Interpretability Models in the Random Forest sRNA
Table of Contents
- PRE-SETUP = We are simply getting the environment and data ready
- TRAIN NEW H2O MODEL = To make use of H2O’s libraries, we need to create the models in H2O (ie. our original RF model cannot be directly converted to an h2o model)
- COMPARE MODEL’S PREDICTIONS = We have the OrigRF and all the other new models generate predictions on the testing data
- COMPARE MODEL’S METRICS = A true validation of the models’ performance, and a correct way of validating if our new models make good proxies for the original model
- RANDOM FOREST EXPLAINER (or PDPs) = We applied RFE to our origRF model
- LIME = Here we actually begin to apply LIME to our RF models
- SHAP Values = We use the H2O library to generate the SHAP values for our H2O RF model
- PDPs = Similar to the previous section, we use the H2O library to generate the PDPs for the H2O RF model
Introduction
The goal of this project was to apply interpretability models to the random forest classifier in the PeerJ article “Prioritizing bona fide bacterial small RNAs with machine learning classifiers” with the purposes of gaining deeper insights into the prioritization of bona fide bacterial sRNAs.
Some key Terms
- OrigRF/Orig RF = Original random forest as found in link
- OrigRF_[scaled/normalized] = refers to any model that was trained in the same manner(same libraries and settings) as in the Original random forest model, but using scaled or normalized data.
- By scaling the data we mean scaling and centering
- By normalizing we mean fit the values between 0 and 1
- RFE = Random Forest Explainer
- RF = Random Forest
- IM = Interpretability Model
- “SS” = the free energy of the predicted secondary structure of the sRNA - mostly negative values
- “Pos10wrtsRNAStart” = distance to their closest predicted promoter site
- “DistTerm” = distance to their closest predicted Rho-independent terminator
- “Distance” = distance to the closest reading frame on the LEFT(“upstream”) side
- “sameStrand” = boolean, if transcription is going in the same direction as the ORF(Left Open Reading Frame)
- ORF = genomic sequence that’s supposed to code for a protein
- “DownDistance” = distance to the closest reading frame on the RIGHT(“downstream”) side
- “sameDownStrand” = boolean, if transcription is going in the same direction as the RORF(Right Open Reading Frame)
Side note:* When used the word original to refer to the RF model created in the base article, or to refer to models that were created(trained) in the same way as the base random forest model.
I) PRE-SETUP
This section is all about getting the libraries and data needed to run this R notebook. We will preprocess the data and load the original model. We start with some OPTIONAL memory cleaning (useful if you want to run this from R Studio and with a clean environment)
# This section is just about getting the environment ready and cleaned up. Since this project was originally written as a regular R script, and worked from within R Studio, it was just usefull in resetting and cleaning up the the environment, and so this block of code is optional
clc <- function() cat("\014") ; clc()
remove(list = ls())
gc() used (Mb) gc trigger (Mb) max used (Mb)
Ncells 2479304 132.5 6565627 350.7 6565627 350.7
Vcells 5037059 38.5 26355833 201.1 65162034 497.21. Install required libraries
The following code is based on the h2o documentation site, as it’s their recommended way for downloading and installing h2o for R, with only the first if being added as to prevent accidental reinstallations. Here is the link for those interested in it.
if (! ("h2o" %in% rownames(installed.packages()))) {
# The following two commands remove any previously installed H2O packages for R.
if ("package:h2o" %in% search()) { detach("package:h2o", unload=TRUE) }
if ("h2o" %in% rownames(installed.packages())) { remove.packages("h2o") }
# Next, we download packages that H2O depends on.
pkgs <- c("RCurl","jsonlite")
for (pkg in pkgs) {
if (! (pkg %in% rownames(installed.packages()))) { install.packages(pkg) }
}
# Now we download, install and initialize the H2O package for R.
install.packages("h2o", type="source", repos="http://h2o-release.s3.amazonaws.com/h2o/rel-zahradnik/4/R")
} The rest of the libraries can be installed in the same way as with any other library.
packages <- c("lime", "ROCR", "PRROC", "randomForest", "randomForestExplainer", "png")
for (package in packages) {
if (! (package %in% rownames(installed.packages()))) { install.packages(package) }
}2. Load the libraries
library("h2o") # ML model building
library("ROCR") # ML evaluation
library("PRROC") # ML evaluation
library("randomForest") # ML model building
library("randomForestExplainer") # ML Global interpretation
library("lime") # ML local interpretation
library("png")3. Load and preprocess the data
3.1 Load the Model’s Training Data
trainDataSet <- read.csv("./DataSets/combinedData.csv", header = TRUE)
trainDataSet[,"Class"] <- as.logical(trainDataSet[,"Class"]) # guarantees that our h2o model trains the random forest (RF) as a classification tree and not a regression tree3.2 Scale and Center the Training Data
dataSetTrain_scaled <- scale(trainDataSet[,-c(5,7,8,9)], center = TRUE, scale = TRUE)
dataSetTrain_scaled <- cbind( dataSetTrain_scaled[,(1:4)],
trainDataSet[,5],
dataSetTrain_scaled[,5],
trainDataSet[,c(7,8,9)])
colnames(dataSetTrain_scaled) <- c("SS", "Pos10wrtsRNAStart", "DistTerm", "Distance",
"sameStrand", "DownDistance", "sameDownStrand", "ID", "Class")
dataSetTrain_scaled[,"Class"] <- as.logical(dataSetTrain_scaled[,"Class"]) # Just making sure Class is taken a logical value
rbind( head(dataSetTrain_scaled,5), tail(dataSetTrain_scaled,5) )3.3 Normalize the Training Data
normalize <- function(x) {
return ((x - min(x)) / (max(x) - min(x)))
}
dataSetTrain_norm <- trainDataSet
dataSetTrain_norm$SS <- normalize(dataSetTrain_norm$SS)
dataSetTrain_norm$Pos10wrtsRNAStart <- normalize(dataSetTrain_norm$Pos10wrtsRNAStart)
dataSetTrain_norm$DistTerm <- normalize(dataSetTrain_norm$DistTerm)
dataSetTrain_norm$Distance <- normalize(dataSetTrain_norm$Distance)
dataSetTrain_norm$DownDistance <- normalize(dataSetTrain_norm$DownDistance)
dataSetTrain_norm[,"Class"] <- as.logical(dataSetTrain_norm[,"Class"]) # Just making sure Class is taken a logical value
rbind( head(dataSetTrain_norm,5), tail(dataSetTrain_norm,5) ) 3.4 Check for normality in the data
There are certain parameter() in the LIME functions that are recommended based on the distribution of the data. Based on these paremeters, we decided to check for normality in the data
dataSetTrainX <- trainDataSet[,-(8:9)]
dataSetTrainY <- trainDataSet[,(8:9)]
hist(x = as.numeric(dataSetTrainX[,1]), main = "Histogram of SS")
qqnorm(as.numeric(dataSetTrainX[,1]))
qqline(as.numeric(dataSetTrainX[,1]), col = "red", lwd = 2)
hist(x = as.numeric(dataSetTrainX[,2]), main = "Histogram of Pos10wrtsRNAStart")
qqnorm(as.numeric(dataSetTrainX[,2]))
qqline(as.numeric(dataSetTrainX[,2]), col = "red", lwd = 2)
hist(x = as.numeric(dataSetTrainX[,3]), main = "Histogram of DistTerm")
qqnorm(as.numeric(dataSetTrainX[,3]))
qqline(as.numeric(dataSetTrainX[,3]), col = "red", lwd = 2)
hist(x = as.numeric(dataSetTrainX[,4]), main = "Histogram of Distance")
qqnorm(as.numeric(dataSetTrainX[,4]))
qqline(as.numeric(dataSetTrainX[,4]), col = "red", lwd = 2)
hist(x = as.numeric(dataSetTrainX[,5]), main = "Histogram of sameStrand")
qqnorm(as.numeric(dataSetTrainX[,5]))
qqline(as.numeric(dataSetTrainX[,5]), col = "red", lwd = 2)
hist(x = as.numeric(dataSetTrainX[,6]), main = "Histogram of DownDistance")
qqnorm(as.numeric(dataSetTrainX[,6]))
qqline(as.numeric(dataSetTrainX[,6]), col = "red", lwd = 2)
hist(x = as.numeric(dataSetTrainX[,7]), main = "Histogram of sameDownStrand")
qqnorm(as.numeric(dataSetTrainX[,7]))
qqline(as.numeric(dataSetTrainX[,7]), col = "red", lwd = 2)
3.5 Load testing Datasets
Load Original Testing Data Sets
slt2dataPos <- read.csv("./DataSets/SLT2_Positives.tsv", sep = "\t", header = TRUE)
slt2dataPos$Class <- rep(1,nrow(slt2dataPos))
slt2dataNeg <- read.csv("./DataSets/SLT2_Negatives.tsv", sep = "\t", header = TRUE)
slt2dataNeg$Class <- rep(0,nrow(slt2dataNeg))
slt2data <- rbind(slt2dataPos,slt2dataNeg)
ludataPos <- read.csv("./DataSets/Lu_Positives.tsv", sep = "\t", header = TRUE)
ludataPos$Class <- rep(1,nrow(ludataPos))
ludataNeg <- read.csv("./DataSets/Lu_Negatives.tsv", sep = "\t", header = TRUE)
ludataNeg$Class <- rep(0,nrow(ludataNeg))
ludata <- rbind(ludataPos,ludataNeg)3.6 Scale testing Datasets
Even though the scale function can scale and center the data for us, we need to scale the testing data sets using the training data’s standard deviation and means to obtain adequate results.
Scaling Function: * Xi = An instance * Xmean = Mean/Average of all samples
* Xsd = Standard Deviation of our data set * Zi = Scaled Value of Xi * Zi = (Xi - Xmean) / Xsd
# Standard Deviation and Mean for Standarization
scale_td <- function( Xi, Xmean, Xsd) {
return ( (Xi - Xmean) / Xsd ) # Zi
}
trainData_sd_SS <- sd(trainDataSet[,"SS"])
trainData_sd_Pos <- sd(trainDataSet[,"Pos10wrtsRNAStart"])
trainData_sd_DisTerm <- sd(trainDataSet[,"DistTerm"])
trainData_sd_Dis <- sd(trainDataSet[,"Distance"])
trainData_sd_DownDis <- sd(trainDataSet[,"DownDistance"])
trainData_mean_SS <- mean(trainDataSet[,"SS"])
trainData_mean_Pos <- mean(trainDataSet[,"Pos10wrtsRNAStart"])
trainData_mean_DisTerm <- mean(trainDataSet[,"DistTerm"])
trainData_mean_Dis <- mean(trainDataSet[,"Distance"])
trainData_mean_DownDis <- mean(trainDataSet[,"DownDistance"])
slt2data_scaled <- slt2data
slt2data_scaled$SS <- scale_td(slt2data_scaled$SS,
trainData_mean_SS, trainData_sd_SS)
slt2data_scaled$Pos10wrtsRNAStart <- scale_td(slt2data_scaled$Pos10wrtsRNAStart,
trainData_mean_Pos, trainData_sd_Pos)
slt2data_scaled$DistTerm <- scale_td(slt2data_scaled$DistTerm,
trainData_mean_DisTerm, trainData_sd_DisTerm)
slt2data_scaled$Distance <- scale_td(slt2data_scaled$Distance,
trainData_mean_Dis, trainData_sd_Dis)
slt2data_scaled$DownDistance <- scale_td(slt2data_scaled$DownDistance,
trainData_mean_DownDis, trainData_sd_DownDis)
ludata_scaled <- ludata
ludata_scaled$SS <- scale_td(ludata_scaled$SS,
trainData_mean_SS, trainData_sd_SS)
ludata_scaled$Pos10wrtsRNAStart <- scale_td(ludata_scaled$Pos10wrtsRNAStart,
trainData_mean_Pos, trainData_sd_Pos)
ludata_scaled$DistTerm <- scale_td(ludata_scaled$DistTerm,
trainData_mean_DisTerm, trainData_sd_DisTerm)
ludata_scaled$Distance <- scale_td(ludata_scaled$Distance,
trainData_mean_Dis, trainData_sd_Dis)
ludata_scaled$DownDistance <- scale_td(ludata_scaled$DownDistance,
trainData_mean_DownDis, trainData_sd_DownDis)
rbind( head(slt2data_scaled,5), tail(slt2data_scaled,5))rbind( head(ludata_scaled,5), tail(ludata_scaled,5))3.7 Normalize testing Datasets
In a similar fashion to what we did in 3.6, we also need to use the training data’s max and min values to properly normalize each feature.
normalize_td <- function(x, min, max) {
return ( (x - min) / (max - min) )
}
trainData_max_SS <- max(trainDataSet[,"SS"])
trainData_max_Pos <- max(trainDataSet[,"Pos10wrtsRNAStart"])
trainData_max_DisTerm <- max(trainDataSet[,"DistTerm"])
trainData_max_Dis <- max(trainDataSet[,"Distance"])
trainData_max_DownDis <- max(trainDataSet[,"DownDistance"])
trainData_min_SS <- min(trainDataSet[,"SS"])
trainData_min_Pos <- min(trainDataSet[,"Pos10wrtsRNAStart"])
trainData_min_DisTerm <- min(trainDataSet[,"DistTerm"])
trainData_min_Dis <- min(trainDataSet[,"Distance"])
trainData_min_DownDis <- min(trainDataSet[,"DownDistance"])
slt2data_norm <- slt2data
slt2data_norm$SS <- normalize_td(slt2data_norm$SS,
trainData_min_SS, trainData_max_SS)
slt2data_norm$Pos10wrtsRNAStart <- normalize_td(slt2data_norm$Pos10wrtsRNAStart,
trainData_min_Pos, trainData_max_Pos)
slt2data_norm$DistTerm <- normalize_td(slt2data_norm$DistTerm,
trainData_min_DisTerm, trainData_max_DisTerm)
slt2data_norm$Distance <- normalize_td(slt2data_norm$Distance,
trainData_min_Dis, trainData_max_Dis)
slt2data_norm$DownDistance <- normalize_td(slt2data_norm$DownDistance,
trainData_min_DownDis, trainData_max_DownDis)
ludata_norm <- ludata
ludata_norm$SS <- normalize_td(ludata_norm$SS,
trainData_min_SS, trainData_max_SS)
ludata_norm$Pos10wrtsRNAStart <- normalize_td(ludata_norm$Pos10wrtsRNAStart,
trainData_min_Pos, trainData_max_Pos)
ludata_norm$DistTerm <- normalize_td(ludata_norm$DistTerm,
trainData_min_DisTerm, trainData_max_DisTerm)
ludata_norm$Distance <- normalize_td(ludata_norm$Distance,
trainData_min_Dis, trainData_max_Dis)
ludata_norm$DownDistance <- normalize_td(ludata_norm$DownDistance,
trainData_min_DownDis, trainData_max_DownDis)
rbind( head(slt2data_norm,5), tail(slt2data_norm,5))rbind( head(ludata_norm,5), tail(ludata_norm,5))4. Load original model
The original article, titled “Prioritizing bona fide bacterial small RNAs with machine learning classifiers”, and the source materials(training data, testing data, R scripts, .rds file, etc.) can all be found in PeerJ under the following link. The original .rds file can also be downloaded from github.
4.1 Load/Retrain Orig Model
We can either retrain the original model, or load the .rds file provided in the original article. For the purpose of saving time, we simply loaded the model.
origRF <- readRDS("RF_classifier4sRNA.rds")4.2 Retrain new models with Scaled and Normalized Data
Since some of the IM models required scaling or normalized data to properly function, we needed to create 2 new models in the same way as the one loaded in 4.1.
tuneRF(dataSetTrainX[,c(1:7)], y = factor(dataSetTrainY[,2]), ntreeTry = 400, mtryStart = 2)mtry = 2 OOB error = 10.74%
Searching left ...
mtry = 1 OOB error = 11.5%
-0.07142857 0.05
Searching right ...
mtry = 4 OOB error = 10.43%
0.02857143 0.05
mtry OOBError
1.OOB 1 0.1150307
2.OOB 2 0.1073620
4.OOB 4 0.1042945
set.seed(1234)
origRF_scaled <- randomForest(x = dataSetTrain_scaled[,c(1:7)], y = factor(dataSetTrainY[,2]), mtry = 2, ntree = 400, importance = TRUE)
origRF_norm <- randomForest(x = dataSetTrain_norm[,c(1:7)], y = factor(dataSetTrainY[,2]), mtry = 2, ntree = 400, importance = TRUE)II) TRAIN THE NEW H2O MODEL
Since we are going to use the h2o library to perform some of our machine learning interpretability, we need to train an equivalent one, using the h2o tools.
5. Intialize H2O and load training data
h2o.init( # Initialize h2o
nthreads = -1, # -1 = use all available threads
max_mem_size = "8G" # specify the amount of memory to use
) Connection successful!
R is connected to the H2O cluster:
H2O cluster uptime: 3 days 10 hours
H2O cluster timezone: America/St_Johns
H2O data parsing timezone: UTC
H2O cluster version: 3.30.0.4
H2O cluster version age: 1 month and 4 days
H2O cluster name: H2O_started_from_R_carlos.salcedo_lix921
H2O cluster total nodes: 1
H2O cluster total memory: 6.82 GB
H2O cluster total cores: 8
H2O cluster allowed cores: 8
H2O cluster healthy: TRUE
H2O Connection ip: localhost
H2O Connection port: 54321
H2O Connection proxy: NA
H2O Internal Security: FALSE
H2O API Extensions: Amazon S3, XGBoost, Algos, AutoML, Core V3, TargetEncoder, Core V4
R Version: R version 3.6.1 (2019-07-05) h2o.removeAll() # clean up the system, h2o-wise (ie. kill any running h2o clusters)
h2o.no_progress()
trainData <- as.h2o(trainDataSet)
trainData_scaled <- as.h2o(dataSetTrain_scaled)
trainData_norm <- as.h2o(dataSetTrain_norm) 6. Build model
6.1 Build model with Orig Training data
rfh2o <- h2o.randomForest(
training_frame = trainData, # training using the normal data
x = 1:7, # features to use to generate the prediction
y = 9, # Class type -> what we want to predict
model_id = "rf1_sRNA", # name of model in h2o
ntrees = 400, # max number of trees
seed = 1234, # seed, has to be set WITHIN the h2o function and it's supposed to be different from "R's seed", so results might not be exactly the same as orig model, but should be similar enough
mtries = 2, # Same as original model
max_depth = 30
)6.2 Build model with Scaled Training data
rfh2o_scaled <- h2o.randomForest(
training_frame = trainData_scaled,
x = 1:7,
y = 9,
model_id = "rf2_sRNA",
ntrees = 400,
seed = 1234,
mtries = 2,
max_depth = 30
)6.3 Build model with Norm Training data
rfh2o_norm <- h2o.randomForest(
training_frame = trainData_norm,
x = 1:7,
y = 9,
model_id = "rf3_sRNA",
ntrees = 400,
seed = 1234,
mtries = 2,
max_depth = 30
)6.4 BONUS: Build GLM model
As we were doing tests with LIME, we discovered that it yielded irregular responses, and as a result, the explanations could not be trusted at the time. It seems to be a known issue that LIME is not perfect for every model, and that it struggles with data that is highly diverse. In this use case, our data contains mixed numerical and categorical features, significantly different scales and magnitudes in our numercial features, and an inbalanced data sets that favors predictions of sRNAs being false. Since lime creates a local interpretable linear model for its explanations, the GLM constructed here was simply to verify the hypothesis that a linear model would be unable to explain the data. The “unbalanced-ness” seems to create models where the features tend always support a result of “false”, while claiming that all the features are against a result of true, which will be evicenced in section 15.1.
glmh2o <- h2o.glm( # https://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/glm.html
training_frame = trainData, # training using the normal data
x = 1:7, # features to use to generate the prediction
y = 9, # Class type -> what we want to predict
model_id = "glm_sRNA", # name of model in h2o
seed = 1234,
family = "binomial"
)
glmh2o_perf <- h2o.performance(glmh2o)
glmh2o_perfH2OBinomialMetrics: glm
** Reported on training data. **
MSE: 0.1476104
RMSE: 0.3842009
LogLoss: 0.4680047
Mean Per-Class Error: 0.2607362
AUC: 0.7639542
AUCPR: 0.5749941
Gini: 0.5279085
R^2: 0.2127447
Residual Deviance: 610.2781
AIC: 626.2781
Confusion Matrix (vertical: actual; across: predicted) for F1-optimal threshold:
Maximum Metrics: Maximum metrics at their respective thresholds
Gains/Lift Table: Extract with `h2o.gainsLift(<model>, <data>)` or `h2o.gainsLift(<model>, valid=<T/F>, xval=<T/F>)`h2o.coef(glmh2o) Intercept SS Pos10wrtsRNAStart DistTerm Distance sameStrand DownDistance sameDownStrand
-5.588791e-02 -1.618068e-02 4.486513e-05 -2.304662e-03 -5.368581e-05 -4.228112e-01 1.958833e-04 2.178087e-01 h2o.coef_norm(glmh2o) Intercept SS Pos10wrtsRNAStart DistTerm Distance sameStrand DownDistance sameDownStrand
-1.31299148 -0.37876153 0.01831303 -0.93619002 -0.02021210 -0.21155895 0.08702437 0.10892590 glmh2o@model$coefficients_tableCoefficients: glm coefficientsh2o.r2(glmh2o) # R^2 value is really low, and having a Max Recall being really low, meaning that LIME is also predicting that everything should be false[1] 0.21274477. Preview H2O’s RF
This is the performance with the OOB error, based on the training process (ie training data)
rfh2oModel Details:
==============
H2OBinomialModel: drf
Model ID: rf1_sRNA
Model Summary:
H2OBinomialMetrics: drf
** Reported on training data. **
** Metrics reported on Out-Of-Bag training samples **
MSE: 0.08046297
RMSE: 0.28366
LogLoss: 0.2701744
Mean Per-Class Error: 0.1247444
AUC: 0.9370883
AUCPR: 0.8592391
Gini: 0.8741767
R^2: 0.5708641
Confusion Matrix (vertical: actual; across: predicted) for F1-optimal threshold:
Maximum Metrics: Maximum metrics at their respective thresholds
Gains/Lift Table: Extract with `h2o.gainsLift(<model>, <data>)` or `h2o.gainsLift(<model>, valid=<T/F>, xval=<T/F>)`rfh2o@model$variable_importancesVariable Importances: h2o.varimp_plot(rfh2o)
rfh2o_training_peformance <- h2o.performance(rfh2o)
rfh2o_training_peformanceH2OBinomialMetrics: drf
** Reported on training data. **
** Metrics reported on Out-Of-Bag training samples **
MSE: 0.08046297
RMSE: 0.28366
LogLoss: 0.2701744
Mean Per-Class Error: 0.1247444
AUC: 0.9370883
AUCPR: 0.8592391
Gini: 0.8741767
R^2: 0.5708641
Confusion Matrix (vertical: actual; across: predicted) for F1-optimal threshold:
Maximum Metrics: Maximum metrics at their respective thresholds
Gains/Lift Table: Extract with `h2o.gainsLift(<model>, <data>)` or `h2o.gainsLift(<model>, valid=<T/F>, xval=<T/F>)`III) COMPARE MODEL’S PREDICTIONS
In this section we’ll get the predictions all the models generate on the testing data LU and SLT2. The goal of this section is not to prove which model is the best, but to prove that the models are similar enough to the point that the analysis of the H2O model will be a valid analogy model for the original RF model.
8. Get Testing Data Predictions from the Models
origRF_lu_pred <- predict(origRF, ludata[,-8], type = "prob")
origRF_slt2_pred <- predict(origRF, slt2data[,-8], type = "prob")
origRF_scaled_lu_pred <- predict(origRF_scaled, ludata_scaled[,-8], type = "prob")
origRF_scaled_slt2_pred <- predict(origRF_scaled, slt2data_scaled[,-8], type = "prob")
origRF_norm_lu_pred <- predict(origRF_norm, ludata_norm[,-8], type = "prob")
origRF_norm_slt2_pred <- predict(origRF_norm, slt2data_norm[,-8], type = "prob")
rfh2o_slt2_pred <- h2o.predict(object = rfh2o, newdata = as.h2o(slt2data[,-8]) )
rfh2o_lu_pred <- h2o.predict(object = rfh2o, newdata = as.h2o(ludata[,-8]) )
rfh2o_scaled_slt2_pred <- h2o.predict(object = rfh2o_scaled, newdata = as.h2o(slt2data_scaled[,-8]) )
rfh2o_scaled_lu_pred <- h2o.predict(object = rfh2o_scaled, newdata = as.h2o(ludata_scaled[,-8]) )
rfh2o_norm_slt2_pred <- h2o.predict(object = rfh2o_norm, newdata = as.h2o(slt2data_norm[,-8]) )
rfh2o_norm_lu_pred <- h2o.predict(object = rfh2o_norm, newdata = as.h2o(ludata_norm[,-8]) )
glm_slt2_pred <- h2o.predict(object = glmh2o, newdata = as.h2o(slt2data[,-8]) )
glm_lu_pred <- h2o.predict(object = glmh2o, newdata = as.h2o(ludata[,-8]) )9. Create a comparison function
This function is to simplify the comparison between the predictions from the Original RF model and the H2O RF model. Later on this, this function will be used to compare the predictions between the OrigRF model and the other models.
# This function will be used later on, and
# a = the original model's predictions
# b = the alternate model's predictions
# c = the correct prediction
compareAnswers <- function(a,b,c){
if ( a == c & b == c )
{ res="BOTH_RIGHT" }
else if ( a == c & b != c )
{ res="OnlyA" }
else if ( a != c & b == c )
{ res="OnlyB" }
else
{ res="BOTH_WRONG" }
return(res)
}10. SLT2 comparisons
10.1 Build SLT2 Predictions table
slt2_predictions <- cbind(as.data.frame(slt2data), # Input
as.data.frame(origRF_slt2_pred[,2]), # Orig RF predictions
as.data.frame(origRF_scaled_slt2_pred[,2]), # Orig RF Scaled predictions
as.data.frame(origRF_norm_slt2_pred[,2]), # Orig RF Normalized predictions
as.data.frame(rfh2o_slt2_pred[,3]), # H2O RF predictions
as.data.frame(rfh2o_scaled_slt2_pred[,3]), # Scaled RF predictions
as.data.frame(rfh2o_norm_slt2_pred[,3]) # Normalized RF predictions
)
colnames(slt2_predictions) <- c("SS", "Pos10wrtsRNAStart", "DistTerm", "Distance",
"sameStrand", "DownDistance", "sameDownStrand", "Class",
"OrigRF_P","OrigRF_scaled_P","OrigRF_norm_P",
"RF_H2O_P","RFH2O_scaled_P","RFH2O_norm_P")
slt2_predictions$origPreds <- ifelse( slt2_predictions$OrigRF_P >= 0.5, 1, 0)
slt2_predictions$origScaledPreds <- ifelse( slt2_predictions$OrigRF_scaled_P >= 0.5, 1, 0)
slt2_predictions$origNormPreds <- ifelse( slt2_predictions$OrigRF_norm_P >= 0.5, 1, 0)
slt2_predictions$h2oPreds <- ifelse( slt2_predictions$RF_H2O_P >= 0.5, 1, 0)
slt2_predictions$h2oScaledPreds <- ifelse( slt2_predictions$RFH2O_scaled_P >= 0.5, 1, 0)
slt2_predictions$h2oNormPreds <- ifelse( slt2_predictions$RFH2O_norm_P >= 0.5, 1, 0)
slt2_predictions$origVsH2O <- NA
slt2_predictions$origVsOrigScaled <- NA
slt2_predictions$origVsOrigNorm <- NA
slt2_predictions$origVsH2OScaled <- NA
slt2_predictions$origVsH2ONorm <- NA
for( i in 1:nrow(slt2_predictions) ){
# Based on the way we are feeding the values to the compareAnswers function,
# Similiarities = OnlyA means that only the Orig RF got the right answer
# Similiarities = OnlyB means that only the other RF got the right answer
# All other answers will tell us where both models were right("BOTH_RIGHT"), or
# wrong ("BOTH_WRONG")
slt2_predictions[i,]$origVsOrigScaled <- compareAnswers(
slt2_predictions[i,]$origPreds, slt2_predictions[i,]$origScaledPreds,slt2_predictions[i,]$Class )
slt2_predictions[i,]$origVsOrigNorm <- compareAnswers(
slt2_predictions[i,]$origPreds, slt2_predictions[i,]$origNormPreds, slt2_predictions[i,]$Class )
slt2_predictions[i,]$origVsH2O <- compareAnswers(
slt2_predictions[i,]$origPreds, slt2_predictions[i,]$h2oPreds, slt2_predictions[i,]$Class )
slt2_predictions[i,]$origVsH2OScaled <- compareAnswers(
slt2_predictions[i,]$origPreds, slt2_predictions[i,]$h2oScaledPreds, slt2_predictions[i,]$Class )
slt2_predictions[i,]$origVsH2ONorm <- compareAnswers(
slt2_predictions[i,]$origPreds, slt2_predictions[i,]$h2oNormPreds, slt2_predictions[i,]$Class )
}
rbind( head(slt2_predictions, 5), tail(slt2_predictions, 5) )str(slt2_predictions)'data.frame': 1986 obs. of 25 variables:
$ SS : num -48.3 -8.8 -37.4 -26.3 -41.7 ...
$ Pos10wrtsRNAStart: int -56 -14 119 -15 -23 -48 -36 77 -15 -14 ...
$ DistTerm : int 0 1000 0 0 1000 1000 112 0 0 0 ...
$ Distance : int 0 -24 -98 -47 -2 -23 -53 -22 -71 -28 ...
$ sameStrand : int 1 0 0 1 0 0 0 0 1 0 ...
$ DownDistance : int 14 0 20 17 1 47 245 0 0 175 ...
$ sameDownStrand : int 1 0 0 1 1 0 0 0 1 0 ...
$ Class : num 1 1 1 1 1 1 1 1 1 1 ...
$ OrigRF_P : num 0.73 0.383 0.775 0.985 0.57 ...
$ OrigRF_scaled_P : num 0.745 0.362 0.72 0.98 0.58 ...
$ OrigRF_norm_P : num 0.743 0.328 0.748 0.98 0.61 ...
$ RF_H2O_P : num 0.74 0.372 0.705 0.973 0.539 ...
$ RFH2O_scaled_P : num 0.746 0.364 0.71 0.98 0.437 ...
$ RFH2O_norm_P : num 0.75 0.375 0.708 0.979 0.438 ...
$ origPreds : num 1 0 1 1 1 1 1 1 1 1 ...
$ origScaledPreds : num 1 0 1 1 1 1 1 1 1 1 ...
$ origNormPreds : num 1 0 1 1 1 1 1 1 1 1 ...
$ h2oPreds : num 1 0 1 1 1 1 1 1 1 1 ...
$ h2oScaledPreds : num 1 0 1 1 0 1 1 1 1 1 ...
$ h2oNormPreds : num 1 0 1 1 0 1 1 1 1 1 ...
$ origVsH2O : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
$ origVsOrigScaled : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
$ origVsOrigNorm : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
$ origVsH2OScaled : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
$ origVsH2ONorm : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...As an additional check, we also reviewed the correlations between the OrigRF predictions, and the alternative models’s predictions with the SLT2 predictions.
cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"RF_H2O_P"]) [1] 0.9842173cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"OrigRF_scaled_P"])[1] 0.9979076cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"OrigRF_norm_P"])[1] 0.9976877cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"RFH2O_scaled_P"]) [1] 0.9846473cor(slt2_predictions[,"OrigRF_P"],slt2_predictions[,"RFH2O_norm_P"])[1] 0.984723310.2 Compare OrigRF vs the other Models with the SLT2 data
Here, we are simply taking a look at how many instances the OrigRF model got correct vs the other models, and how many instances both models predicted correctly or wrong. * A value of OnlyA means that only the orig RF got the right answer * A value of OnlyB means that only the other RF got the right answer * All other answers will tell us where both models were right (“BOTH_RIGHT”) or wrong (“BOTH_WRONG”)
10.2.2 Compare OrigRF vs OrigRF Scaled Model
nrow( slt2_predictions[slt2_predictions$origVsOrigScaled == "OnlyA",] )[1] 8nrow( slt2_predictions[slt2_predictions$origVsOrigScaled == "OnlyB",] )[1] 8nrow( slt2_predictions[slt2_predictions$origVsOrigScaled == "BOTH_WRONG",] )[1] 161nrow( slt2_predictions[slt2_predictions$origVsOrigScaled == "BOTH_RIGHT",] )[1] 180910.2.3 Compare OrigRF vs OrigRF Norm Model
nrow( slt2_predictions[slt2_predictions$origVsOrigNorm == "OnlyA",] )[1] 8nrow( slt2_predictions[slt2_predictions$origVsOrigNorm == "OnlyB",] )[1] 12nrow( slt2_predictions[slt2_predictions$origVsOrigNorm == "BOTH_WRONG",] )[1] 157nrow( slt2_predictions[slt2_predictions$origVsOrigNorm == "BOTH_RIGHT",] )[1] 180910.2.4 Compare OrigRF vs H2O Model
nrow( slt2_predictions[slt2_predictions$origVsH2O == "OnlyA",] )[1] 26nrow( slt2_predictions[slt2_predictions$origVsH2O == "OnlyB",] )[1] 19nrow( slt2_predictions[slt2_predictions$origVsH2O == "BOTH_WRONG",] )[1] 150nrow( slt2_predictions[slt2_predictions$origVsH2O == "BOTH_RIGHT",] )[1] 179110.2.5 Compare OrigRF vs H2O Model Scaled
nrow( slt2_predictions[slt2_predictions$origVsH2OScaled == "OnlyA",] )[1] 29nrow( slt2_predictions[slt2_predictions$origVsH2OScaled == "OnlyB",] )[1] 19nrow( slt2_predictions[slt2_predictions$origVsH2OScaled == "BOTH_WRONG",] )[1] 150nrow( slt2_predictions[slt2_predictions$origVsH2OScaled == "BOTH_RIGHT",] )[1] 178810.2.6 Compare OrigRF vs H2O Model Normalized
nrow( slt2_predictions[slt2_predictions$origVsH2ONorm == "OnlyA",] )[1] 32nrow( slt2_predictions[slt2_predictions$origVsH2ONorm == "OnlyB",] )[1] 19nrow( slt2_predictions[slt2_predictions$origVsH2ONorm == "BOTH_WRONG",] )[1] 150nrow( slt2_predictions[slt2_predictions$origVsH2ONorm == "BOTH_RIGHT",] )[1] 178511. LU Comparisons
11.1 Build LU Predictions table
lu_predictions <- cbind(as.data.frame(ludata), # Input
as.data.frame(origRF_lu_pred[,2]), # Orig RF predictions
as.data.frame(origRF_scaled_lu_pred[,2]), # Orig RF Scaled predictions
as.data.frame(origRF_norm_lu_pred[,2]), # Orig RF Normalized predictions
as.data.frame(rfh2o_lu_pred[,3]), # H2O RF predictions
as.data.frame(rfh2o_scaled_lu_pred[,3]), # Scaled RF predictions
as.data.frame(rfh2o_norm_lu_pred[,3]) # Normalized RF predictions
)
colnames(lu_predictions) <- c("SS", "Pos10wrtsRNAStart", "DistTerm", "Distance",
"sameStrand", "DownDistance", "sameDownStrand", "Class",
"OrigRF_P","OrigRF_scaled_P","OrigRF_norm_P",
"RF_H2O_P","RFH2O_scaled_P","RFH2O_norm_P")
lu_predictions$origPreds <- ifelse( lu_predictions$OrigRF_P >= 0.5, 1, 0)
lu_predictions$origScaledPreds <- ifelse( lu_predictions$OrigRF_scaled_P >= 0.5, 1, 0)
lu_predictions$origNormPreds <- ifelse( lu_predictions$OrigRF_norm_P >= 0.5, 1, 0)
lu_predictions$h2oPreds <- ifelse( lu_predictions$RF_H2O_P >= 0.5, 1, 0)
lu_predictions$h2oScaledPreds <- ifelse( lu_predictions$RFH2O_scaled_P >= 0.5, 1, 0)
lu_predictions$h2oNormPreds <- ifelse( lu_predictions$RFH2O_norm_P >= 0.5, 1, 0)
lu_predictions$origVsH2O <- NA
lu_predictions$origVsOrigScaled <- NA
lu_predictions$origVsOrigNorm <- NA
lu_predictions$origVsH2OScaled <- NA
lu_predictions$origVsH2ONorm <- NA
for( i in 1:nrow(lu_predictions) ){
# Based on the way we are feeding the values to the compareAnswers function,
# Similiarities = OnlyA means that only the Orig RF got the right answer
# Similiarities = OnlyB means that only the other RF got the right answer
# All other answers will tell us where both models were right("BOTH_RIGHT"), or
# wrong ("BOTH_WRONG")
lu_predictions[i,]$origVsOrigScaled <- compareAnswers(
lu_predictions[i,]$origPreds, lu_predictions[i,]$origScaledPreds,lu_predictions[i,]$Class )
lu_predictions[i,]$origVsOrigNorm <- compareAnswers(
lu_predictions[i,]$origPreds, lu_predictions[i,]$origNormPreds, lu_predictions[i,]$Class )
lu_predictions[i,]$origVsH2O <- compareAnswers(
lu_predictions[i,]$origPreds, lu_predictions[i,]$h2oPreds, lu_predictions[i,]$Class )
lu_predictions[i,]$origVsH2OScaled <- compareAnswers(
lu_predictions[i,]$origPreds, lu_predictions[i,]$h2oScaledPreds, lu_predictions[i,]$Class )
lu_predictions[i,]$origVsH2ONorm <- compareAnswers(
lu_predictions[i,]$origPreds, lu_predictions[i,]$h2oNormPreds, lu_predictions[i,]$Class )
}
str(lu_predictions)'data.frame': 3009 obs. of 25 variables:
$ SS : num -174.1 -30.9 -117.6 -30 -33.1 ...
$ Pos10wrtsRNAStart: int -18 -1000 -24 -26 56 -94 -35 12 77 244 ...
$ DistTerm : int 14 445 36 1000 1000 9 6 494 19 1000 ...
$ Distance : int -14 -1217 -292 -121 -112 -32 -83 -65 0 -121 ...
$ sameStrand : int 1 0 0 0 0 1 1 1 1 1 ...
$ DownDistance : int 68 379 18 228 61 74 71 41 0 131 ...
$ sameDownStrand : int 0 1 1 1 1 1 1 1 1 0 ...
$ Class : num 1 1 1 1 1 1 1 1 1 1 ...
$ OrigRF_P : num 0.968 0.295 0.892 0.855 0.695 ...
$ OrigRF_scaled_P : num 0.948 0.305 0.897 0.895 0.72 ...
$ OrigRF_norm_P : num 0.958 0.285 0.91 0.83 0.738 ...
$ RF_H2O_P : num 0.888 0.286 0.879 0.869 0.648 ...
$ RFH2O_scaled_P : num 0.882 0.301 0.877 0.849 0.613 ...
$ RFH2O_norm_P : num 0.885 0.302 0.87 0.851 0.622 ...
$ origPreds : num 1 0 1 1 1 1 1 1 0 0 ...
$ origScaledPreds : num 1 0 1 1 1 1 1 1 0 0 ...
$ origNormPreds : num 1 0 1 1 1 1 1 1 0 0 ...
$ h2oPreds : num 1 0 1 1 1 1 1 1 0 0 ...
$ h2oScaledPreds : num 1 0 1 1 1 1 1 1 0 0 ...
$ h2oNormPreds : num 1 0 1 1 1 1 1 1 0 0 ...
$ origVsH2O : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
$ origVsOrigScaled : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
$ origVsOrigNorm : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
$ origVsH2OScaled : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...
$ origVsH2ONorm : chr "BOTH_RIGHT" "BOTH_WRONG" "BOTH_RIGHT" "BOTH_RIGHT" ...rbind( head(lu_predictions, 5), tail(lu_predictions, 5))As an additional check, we also review the correlations between the OrigRF predictions, and the alternative models’ predictions with the LU data.
cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"RF_H2O_P"]) [1] 0.9870177cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"OrigRF_scaled_P"])[1] 0.9977671cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"OrigRF_norm_P"])[1] 0.9980152cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"RFH2O_scaled_P"]) [1] 0.9863546cor(lu_predictions[,"OrigRF_P"],lu_predictions[,"RFH2O_norm_P"])[1] 0.986641311.2 Compare OrigRF vs the other Models with the LU data
Basically, the same explanation as in section 10.2 * A value of OnlyA means that only the orig RF got the right answer * A value of OnlyB means that only the other RF got the right answer * All other answers will tell us where both models were right (“BOTH_RIGHT”) or wrong (“BOTH_WRONG”)
11.2.1 Compare OrigRF vs OrigRF Scaled Model
nrow( lu_predictions[lu_predictions$origVsOrigScaled == "OnlyA",] )[1] 9nrow( lu_predictions[lu_predictions$origVsOrigScaled == "OnlyB",] )[1] 19nrow( lu_predictions[lu_predictions$origVsOrigScaled == "BOTH_WRONG",] )[1] 398nrow( lu_predictions[lu_predictions$origVsOrigScaled == "BOTH_RIGHT",] )[1] 258311.2.2 Compare OrigRF vs OrigRF Norm Model
nrow( lu_predictions[lu_predictions$origVsOrigNorm == "OnlyA",] )[1] 13nrow( lu_predictions[lu_predictions$origVsOrigNorm == "OnlyB",] )[1] 20nrow( lu_predictions[lu_predictions$origVsOrigNorm == "BOTH_WRONG",] )[1] 397nrow( lu_predictions[lu_predictions$origVsOrigNorm == "BOTH_RIGHT",] )[1] 257911.2.3 Compare OrigRF vs H2O Model
nrow( lu_predictions[lu_predictions$origVsH2O == "OnlyA",] )[1] 35nrow( lu_predictions[lu_predictions$origVsH2O == "OnlyB",] )[1] 33nrow( lu_predictions[lu_predictions$origVsH2O == "BOTH_WRONG",] )[1] 384nrow( lu_predictions[lu_predictions$origVsH2O == "BOTH_RIGHT",] )[1] 255711.2.4 Compare OrigRF vs H2O Model Scaled
nrow( lu_predictions[lu_predictions$origVsH2OScaled == "OnlyA",] )[1] 31nrow( lu_predictions[lu_predictions$origVsH2OScaled == "OnlyB",] )[1] 35nrow( lu_predictions[lu_predictions$origVsH2OScaled == "BOTH_WRONG",] )[1] 382nrow( lu_predictions[lu_predictions$origVsH2OScaled == "BOTH_RIGHT",] )[1] 256111.2.5 Compare OrigRF vs H2O Model Normalized
nrow( lu_predictions[lu_predictions$origVsH2ONorm == "OnlyA",] )[1] 29nrow( lu_predictions[lu_predictions$origVsH2ONorm == "OnlyB",] )[1] 35nrow( lu_predictions[lu_predictions$origVsH2ONorm == "BOTH_WRONG",] )[1] 382nrow( lu_predictions[lu_predictions$origVsH2ONorm == "BOTH_RIGHT",] )[1] 2563IV) COMPARE MODEL’S METRICS
While sections 11.2 and 10.2 suggest that all the models are behaving similarly, making a judgement solely on accuracy can be deceiving. For this reason, we also decided to dive deeper into the model’s performances, and do additional side by side comparison of other performance metrics.
12. Get Performance Metrics
This evaluateData function was copied directly from the source materials(the additional material included in the original article), as it was the one used to evaluate the original model, and we’ve simply added comments a few extra comments. The H2O library already contains tools to obtain performance metrics from its models, so no additional functions were necessary with the H2O ML models.
evaluateData <- function(RF, data, labels){
require(ROCR)
require(PRROC)
require(randomForest)
res <- list()
res$predD <- predict(RF, data, type = "prob") # Predictions as Generated by the model
res$pred <- prediction(res$predD[,2], as.logical(labels)) # Predictions, but converted to S4 Object
res$PR <- performance(res$pred, measure = "prec", x.measure = "rec") # Precision Recall Curve
res$SS <- performance(res$pred, measure="sens", x.measure="spec") # Sensitivity vs. Specificity
res$auc <- performance(res$pred, measure = "auc") # Sensitivity vs. Specificity AUC
res$acc <- performance(res$pred, measure = "acc") # Accuracy
res$pr <- pr.curve(scores.class0 = res$predD[labels == 1,2], # Precision vs. Recall
scores.class1 = res$predD[labels == 0,2],
curve = T)
return(res)
}
# OrigRF Performance Metrics
origRF_slt2_performance <- evaluateData(origRF, slt2data[,-8], slt2data[,8])
origRF_scaled_slt2_performance <- evaluateData(origRF_scaled, slt2data_scaled[,-8], slt2data_scaled[,8])
origRF_norm_slt2_performance <- evaluateData(origRF_norm, slt2data_norm[,-8], slt2data_norm[,8])
origRF_lu_performance <- evaluateData(origRF, ludata[,-8], ludata[,8])
origRF_scaled_lu_performance <- evaluateData(origRF_scaled, ludata_scaled[,-8], ludata_scaled[,8])
origRF_norm_lu_performance <- evaluateData(origRF_norm, ludata_norm[,-8], ludata_norm[,8])
# H2O RF Performance Metrics
# H2O provides a way to retrieve most of the desired metrics
# http://docs.h2o.ai/h2o/latest-stable/h2o-r/docs/reference/h2o.metric.html
slt2data_h2o <- slt2data
slt2data_h2o[,"Class"] <- as.logical(slt2data_h2o[,"Class"])
rfh2o_slt2_performance <- h2o.performance(rfh2o, newdata = as.h2o(slt2data_h2o))
ludata_h2o <- ludata
ludata_h2o[,"Class"] <- as.logical(ludata_h2o[,"Class"])
rfh2o_lu_performance <- h2o.performance(rfh2o, newdata = as.h2o(ludata_h2o))
slt2data_h2o_scaled <- slt2data_scaled
slt2data_h2o_scaled[,"Class"] <- as.logical(slt2data_h2o_scaled[,"Class"])
rfh2o_slt2_performance_scaled <- h2o.performance(rfh2o, newdata = as.h2o(slt2data_h2o_scaled))
ludata_h2o_scaled <- ludata_scaled
ludata_h2o_scaled[,"Class"] <- as.logical(ludata_h2o_scaled[,"Class"])
rfh2o_lu_performance_scaled <- h2o.performance(rfh2o, newdata = as.h2o(ludata_h2o_scaled))
slt2data_h2o_norm <- slt2data_norm
slt2data_h2o_norm[,"Class"] <- as.logical(slt2data_h2o_norm[,"Class"])
rfh2o_slt2_performance_norm <- h2o.performance(rfh2o, newdata = as.h2o(slt2data_h2o_norm))
ludata_h2o_norm <- ludata_norm
ludata_h2o_norm[,"Class"] <- as.logical(ludata_h2o_norm[,"Class"])
rfh2o_lu_performance_norm <- h2o.performance(rfh2o, newdata = as.h2o(ludata_h2o_norm))13. Compare Metrics
13.1 Accuracy
13.1.1 Accuracy Table
metrics_table <- data.frame("Accuracy" = 1:12)
rownames(metrics_table) <- c("origRF_slt2_perf", "origRF_scaled_slt2_perf", "origRF_norm_slt2_perf",
"rfh2o_slt2_perf", "rfh2o_slt2_perf_scaled", "rfh2o_slt2_perf_norm",
"origRF_lu_perf", "origRF_scaled_lu_perf", "origRF_norm_lu_perf",
"rfh2o_lu_perf", "rfh2o_lu_perf_scaled", "rfh2o_lu_perf_norm")
metrics_table["origRF_slt2_perf","Accuracy"] <- nrow(slt2_predictions[slt2_predictions$origVsOrigScaled == "BOTH_RIGHT" | slt2_predictions$origVsOrigScaled == "OnlyA",] )/
nrow(slt2_predictions)
metrics_table["origRF_scaled_slt2_perf","Accuracy"] <- nrow(slt2_predictions[slt2_predictions$origVsOrigScaled == "BOTH_RIGHT" | slt2_predictions$origVsOrigScaled == "OnlyB",] )/
nrow(slt2_predictions)
metrics_table["origRF_norm_slt2_perf","Accuracy"] <- nrow(slt2_predictions[slt2_predictions$origVsOrigNorm == "BOTH_RIGHT" | slt2_predictions$origVsOrigNorm == "OnlyB",] )/
nrow(slt2_predictions)
metrics_table["rfh2o_slt2_perf","Accuracy"] <- nrow(slt2_predictions[slt2_predictions$origVsH2O == "BOTH_RIGHT" | slt2_predictions$origVsH2O == "OnlyB",] )/
nrow(slt2_predictions)
metrics_table["rfh2o_slt2_perf_scaled","Accuracy"] <- nrow(slt2_predictions[slt2_predictions$origVsH2OScaled == "BOTH_RIGHT" | slt2_predictions$origVsH2OScaled == "OnlyB",])/
nrow(slt2_predictions)
metrics_table["rfh2o_slt2_perf_norm","Accuracy"] <- nrow(slt2_predictions[slt2_predictions$origVsH2ONorm == "BOTH_RIGHT" | slt2_predictions$origVsH2ONorm == "OnlyB",])/
nrow(slt2_predictions)
metrics_table["origRF_lu_perf","Accuracy"] <- nrow(lu_predictions[lu_predictions$origVsH2O == "BOTH_RIGHT" | lu_predictions$origVsH2O == "OnlyA",] )/
nrow(lu_predictions)
metrics_table["origRF_scaled_lu_perf","Accuracy"] <- nrow(lu_predictions[lu_predictions$origVsOrigScaled == "BOTH_RIGHT" | lu_predictions$origVsOrigScaled == "OnlyB",] )/
nrow(lu_predictions)
metrics_table["origRF_norm_lu_perf","Accuracy"] <- nrow(lu_predictions[lu_predictions$origVsOrigNorm == "BOTH_RIGHT" | lu_predictions$origVsOrigNorm == "OnlyB",] )/
nrow(lu_predictions)
metrics_table["rfh2o_lu_perf","Accuracy"] <- nrow(lu_predictions[lu_predictions$origVsH2O == "BOTH_RIGHT" | lu_predictions$origVsH2O == "OnlyB",] )/
nrow(lu_predictions)
metrics_table["rfh2o_lu_perf_scaled","Accuracy"] <- nrow(lu_predictions[lu_predictions$origVsH2OScaled == "BOTH_RIGHT" | lu_predictions$origVsH2OScaled == "OnlyB",])/
nrow(lu_predictions)
metrics_table["rfh2o_lu_perf_norm","Accuracy"] <- nrow(lu_predictions[lu_predictions$origVsH2ONorm == "BOTH_RIGHT" | lu_predictions$origVsH2ONorm == "OnlyB",] )/
nrow(lu_predictions)
metrics_table13.1.2 Accuracy graphs based on different thresholds(“cutoff”)
The Legend for the graphs: * Blue = OrigRF * Magenta = OrigRF but scaled * Black = OrigRF but normalized * Red = H2O RF model * Brown = H2O RF model with scaled data * Green = H2O RF model with normalized data
# SLT2 Accuracy Plot
plot(origRF_slt2_performance$acc, col = "blue", lwd = 5, main = "SLT2 Accuracy")
lines(as.double(unlist(origRF_scaled_slt2_performance$acc@x.values)), as.double(unlist(origRF_scaled_slt2_performance$acc@y.values)), col = "magenta", lwd = 4) lines(as.double(unlist(origRF_norm_slt2_performance$acc@x.values)), as.double(unlist(origRF_norm_slt2_performance$acc@y.values)), col = "black", lwd = 1)
lines(h2o.accuracy(rfh2o_slt2_performance), type = "l", col = "red", lwd = 3)lines(h2o.accuracy(rfh2o_slt2_performance_scaled), type = "l", col = "brown", lwd = 3)
lines(h2o.accuracy(rfh2o_slt2_performance_norm), type = "l", col = "green", lwd = 3)
# LU Accuracy Plot
plot(origRF_lu_performance$acc, col = "blue", lwd = 5, main = "LU Accuracy")
lines(as.double(unlist(origRF_scaled_lu_performance$acc@x.values)), as.double(unlist(origRF_scaled_lu_performance$acc@y.values)), col = "magenta", lwd = 4) lines(as.double(unlist(origRF_norm_lu_performance$acc@x.values)), as.double(unlist(origRF_norm_lu_performance$acc@y.values)), col = "black", lwd = 1)
lines(h2o.accuracy(rfh2o_lu_performance), type = "l", col = "red", lwd = 3)lines(h2o.accuracy(rfh2o_lu_performance_scaled), type = "l", col = "brown", lwd = 3)
lines(h2o.accuracy(rfh2o_lu_performance_norm), type = "l", col = "green", lwd = 3)
13.2 AUCPR
The Legend for the graphs: * Blue = OrigRF * Magenta = OrigRF but scaled * Black = OrigRF but normalized * Red = H2O RF model * Brown = H2O RF model with scaled data * Green = H2O RF model with normalized data
metrics_table["origRF_slt2_perf","AUCPR"] <- origRF_slt2_performance$pr$auc.integral
metrics_table["origRF_scaled_slt2_perf","AUCPR"] <- origRF_scaled_slt2_performance$pr$auc.integral
metrics_table["origRF_norm_slt2_perf","AUCPR"] <- origRF_norm_slt2_performance$pr$auc.integral
metrics_table["rfh2o_slt2_perf", "AUCPR"] <- h2o.aucpr(rfh2o_slt2_performance)
metrics_table["rfh2o_slt2_perf_scaled", "AUCPR"] <- h2o.aucpr(rfh2o_slt2_performance_scaled)
metrics_table["rfh2o_slt2_perf_norm", "AUCPR"] <- h2o.aucpr(rfh2o_slt2_performance_norm)
metrics_table["origRF_lu_perf", "AUCPR"] <- origRF_lu_performance$pr$auc.integral
metrics_table["origRF_scaled_lu_perf", "AUCPR"] <- origRF_scaled_lu_performance$pr$auc.integral
metrics_table["origRF_norm_lu_perf", "AUCPR"] <- origRF_norm_lu_performance$pr$auc.integral
metrics_table["rfh2o_lu_perf", "AUCPR"] <- h2o.aucpr(rfh2o_lu_performance)
metrics_table["rfh2o_lu_perf_scaled", "AUCPR"] <- h2o.aucpr(rfh2o_lu_performance_scaled)
metrics_table["rfh2o_lu_perf_norm", "AUCPR"] <- h2o.aucpr(rfh2o_lu_performance_norm)
metrics_tableplot(origRF_slt2_performance$PR, col = "blue", lwd = 5, main = "SLT2 PR Curve" )
lines(as.double(unlist(origRF_scaled_slt2_performance$PR@x.values)), as.double(unlist(origRF_scaled_slt2_performance$PR@y.values)), col = "magenta", lwd = 4 )lines(as.double(unlist(origRF_norm_slt2_performance$PR@x.values)), as.double(unlist(origRF_norm_slt2_performance$PR@y.values)), col = "black", lwd = 2 )
lines(x = h2o.recall(rfh2o_slt2_performance)[,"tpr"],
y = h2o.precision(rfh2o_slt2_performance)[,"precision"],
col = "red", type = "l", lwd = 3
)lines(x = h2o.recall(rfh2o_slt2_performance_scaled)[,"tpr"],
y = h2o.precision(rfh2o_slt2_performance_scaled)[,"precision"],
col = "brown", type = "l", lwd = 3
)
lines(x = h2o.recall(rfh2o_slt2_performance_norm)[,"tpr"],
y = h2o.precision(rfh2o_slt2_performance_norm)[,"precision"],
col = "green", type = "l", lwd = 3
)
plot(origRF_lu_performance$PR, col = "blue", lwd = 5, main = "LU PR Curve" )
lines(as.double(unlist(origRF_scaled_lu_performance$PR@x.values)), as.double(unlist(origRF_scaled_lu_performance$PR@y.values)), col = "magenta", lwd = 4 )lines(as.double(unlist(origRF_norm_lu_performance$PR@x.values)), as.double(unlist(origRF_norm_lu_performance$PR@y.values)), col = "black", lwd = 2 )
lines(x = h2o.recall(rfh2o_lu_performance)[,"tpr"],
y = h2o.precision(rfh2o_lu_performance)[,"precision"],
col = "red", type = "l", lwd = 3
)lines(x = h2o.recall(rfh2o_lu_performance_scaled)[,"tpr"],
y = h2o.precision(rfh2o_lu_performance_scaled)[,"precision"],
col = "brown", type = "l", lwd = 3
)
lines(x = h2o.recall(rfh2o_lu_performance_norm)[,"tpr"],
y = h2o.precision(rfh2o_lu_performance_norm)[,"precision"],
col = "green", type = "l", lwd = 3
)
13.3 Sensitivy vs Specificity Graph
The Legend for the graphs: * Blue = OrigRF * Magenta = OrigRF but scaled * Black = OrigRF but normalized * Red = H2O RF model * Brown = H2O RF model with scaled data * Green = H2O RF model with normalized data
plot(origRF_slt2_performance$SS, col = "blue", lwd = 5, main = "SLT2 Sensitivity vs Specificity")
lines(as.double(unlist(origRF_scaled_slt2_performance$SS@x.values)), as.double(unlist(origRF_scaled_slt2_performance$SS@y.values)), col = "magenta", lwd = 4 )lines(as.double(unlist(origRF_norm_slt2_performance$SS@x.values)), as.double(unlist(origRF_norm_slt2_performance$SS@y.values)), col = "black", lwd = 2 )
lines(x = h2o.specificity(rfh2o_slt2_performance)[,"tnr"],
y = h2o.sensitivity(rfh2o_slt2_performance)[,"tpr"],
col = "red", type = "l", lwd = 3
)lines(x = h2o.specificity(rfh2o_slt2_performance_scaled)[,"tnr"],
y = h2o.sensitivity(rfh2o_slt2_performance_scaled)[,"tpr"],
col = "brown", type = "l", lwd = 3
)
lines(x = h2o.specificity(rfh2o_slt2_performance_norm)[,"tnr"],
y = h2o.sensitivity(rfh2o_slt2_performance_norm)[,"tpr"],
col = "green", type = "l", lwd = 3
)
plot(origRF_lu_performance$SS, col = "blue", lwd = 5, main = "LU Sensitivity vs Specificity")
lines(as.double(unlist(origRF_scaled_lu_performance$SS@x.values)), as.double(unlist(origRF_scaled_lu_performance$SS@y.values)), col = "magenta", lwd = 4 )lines(as.double(unlist(origRF_norm_lu_performance$SS@x.values)), as.double(unlist(origRF_norm_lu_performance$SS@y.values)), col = "black", lwd = 2 )
lines(x = h2o.specificity(rfh2o_lu_performance)[,"tnr"],
y = h2o.sensitivity(rfh2o_lu_performance)[,"tpr"],
col = "red", type = "l", lwd = 3
)lines(x = h2o.specificity(rfh2o_lu_performance_scaled)[,"tnr"],
y = h2o.sensitivity(rfh2o_lu_performance_scaled)[,"tpr"],
col = "brown", type = "l", lwd = 3
)
lines(x = h2o.specificity(rfh2o_lu_performance_norm)[,"tnr"],
y = h2o.sensitivity(rfh2o_lu_performance_norm)[,"tpr"],
col = "green", type = "l", lwd = 3
)
13.4 Other H2O Metrics
Some additional information that’s easily obtainable from the H2O library.
h2o.confusionMatrix(rfh2o_slt2_performance)Confusion Matrix (vertical: actual; across: predicted) for max f1 @ threshold = 0.644166666567326:h2o.confusionMatrix(rfh2o_slt2_performance_scaled)Confusion Matrix (vertical: actual; across: predicted) for max f1 @ threshold = 0.232701931446791:h2o.confusionMatrix(rfh2o_slt2_performance_norm)Confusion Matrix (vertical: actual; across: predicted) for max f1 @ threshold = 0.266167617663741:plot(h2o.F1(rfh2o_slt2_performance), col = "blue", main = "F1 Performance")
lines(h2o.F1(rfh2o_slt2_performance_scaled), col = "red")lines(h2o.F1(rfh2o_slt2_performance_norm), col = "orange")
plot(rfh2o_slt2_performance, # REMINDER: TPR = Sensitivity, FPR = (1 - specificity)
type = "roc",
col = "red",
cex = 0.2,
pch = 10
)
h2o.confusionMatrix(rfh2o_lu_performance)Confusion Matrix (vertical: actual; across: predicted) for max f1 @ threshold = 0.496151700350456:plot(h2o.F1(rfh2o_lu_performance))
plot(rfh2o_lu_performance,
type = "roc",
col = "red",
cex = 0.2,
pch = 10
)
V) RFE/PDPs
Random Forest Explainer(RFE) is a global interpretability model that ultimately allows you to better understand your RF model and create visually attractive and useful PDP plots. The methodology for behind RFE is simple: identify the top most important features, their interactions, and then graph these interactions in what is essentially a 2D PDP plot. Unfortunately, when trying to use RFE within the notebook, the outputs generated errors and the plots were simply not showing. To get around this problem, we will simply add the graphs as generated by the RFE library. The code that generated these graphs be found in here, and the output html generated by the explain_forest() function can also be found “./Your_forest_explained.html”. Keep in mind that the HTML file is automatically generated by the library and does not include all the plots we generated (it seems to only show what it considers the main plots). Since the you still have to generate the plots yourself, the HTML file is a cool bonus, but not a necessary.
14. Variable Importance Measures
14.1 Distribution of Minimal Depth and its Mean
Distribution of minimal depth and its mean
14.2 Importance Measures Table
rfe_importance_frame <- read.csv("./Results/RFE_importance_frame_origRF.csv", sep = ",", header = TRUE)
rfe_importance_frame 14.3 Multiway Importance Plots
14.3.1 Times a root vs Mean min depth vs Number of Nodes
Times a root vs Mean min depth vs Number of Nodes
14.3.2 Accuracy Decrease vs. Gini Decrease vs. Times a Root
Accuracy Decrease vs. Gini Decrease vs. Times a Root
15. GGpairs Comparisons
The following graphs are simply helpful in identifying feature importance 
ggparis main features
ggpairs importance rankings
16. Feature Interactions
16.1 Top 30 Feature Interactions
Top 30 Feature Interactions
From this graph, we gather that the top 5 feature interactions are: 1. DownDistance:SS 1. Distance:SS 1. DownDistance:Pos10wrtsRNAStart, 1. DistTerm:SS, 1. Distance:Pos10wrtsRNAStart Even though Distance and DownDistance were considered the most important features, their interactions did not appear to be as significant as the others
16.2 DownDistance vs. SS
* The higher(closer to 0) the SS, the higher the confidence in the prediction
16.3 Distance vs. SS
* distances greater than -850 => higher chances of having a bona fide sRNA * SS higher than -10 => Lower the prob of having a bona fide sRNA
16.4 DownDistance vs. Pos10wrtsRNAStart
* No clear disernable takeaway
16.5 DistTerm vs. SS
* SS of 0 seems to virtually guarantee that we don’t have a bona fide sRNA
16.6 Distance vs. Pos10wrtsRNAStart
* Having a Pos10wrtsRNAStart near 0 or slightly lower seems to make predictions fuzzy (near 0.5)
16.7 Distance vs. DownDistance
* These are the 2 top features, but their interactions are not among the top ones * When both features are close to 0, the chances of having a bona fide sRNA greatly increase dev.off()
16.8 DownDistance vs. DistTerm
Based on the previous plots, generating the following plots seem like a smart thing to do 
* having a DistTerm less than 50, or close to 0, increases the chances of having a bona fide sRNA
16.9 Distance vs. DistTerm
Distance vs. DistTerm
- having a Distance close to 0 increases the chances of having a bona fide sRNA
16.10 sameStrand vs sameDownStrand
sameStrand vs sameDownStrand
- sameStrand and sameDownStrand were considered the least relevant/important features. However, the overall tone of the graph suggests that having these variables helps boosting the model’s confidence in any given prediction. (ie. compare the tones of the previous graph to this one, and it’s obvious the model has more “confidence” with its overall decisions in this later graph.)
17. RFE Conclusions
There are many conclusions that may be gathered or obtained from the plots above, but the main takeaways can be summarized as follows: * The closer the SS is to 0, the higher the model’s confidence in its decision * Having a Distance between -850 and 0 greatly increases the chances of having an sRNA * An SS of -12 or higher will almost guarantee a non bona fide sRNA * There is a sweet spot for finding bona fide sRNAs if the DownDistance is less than 500 and the Distance is less than 800 * Having a DownDistance less than 60 also greatly increases the chances of having an sRNA * While sameStrand and sameDownStrand were considered the least important features, they seem to contribute in increasing the model’s confidence (at least in classifying something as not being an sRNA, as the graph seems to contain mostly solid blue colors)
NOTE/DISCLAIMER: The numbers here are human approximations based solely on looking at the graphs generated by RFE, and RFE uses only the training data to generate its graphs. In other words, while the goal of the RF is to have a general model applicable to any possible bacterial sRNA, there is no guarantee that these conclusions apply to every species of bacteria in the world.
The RFE library, seems to use many of the pre-existing functions in the randomForest library, combined with ggplot, and some other additional libraries to create better looking and easy to understand graphs. Even though the results are not all that different from the ones we got from RFE, we also decided to use the PDP library and save a comparison between the RFE and PDP graphs in pdp_vs_rfe.pdf. We also used the PDP library to generate partial dependance plot individually for each of the 7 features, but the results were not as positive as we had hope in the sense that we didn’t get any real insights into sRNAs or our model. The graphs for these individual PDP plots can be found in pdp_individual_features.pdf.
VI) LIME
LIME FUNCTION ARGUMENTS
- x The training data used for training the model that should be explained.
- model The model whose output should be explained
- bin_continuous Should continuous variables be binned when making the explanation
- n_bins The number of bins for continuous variables if bin_continuous = TRUE
- quantile_bins Should the bins be based on n_bins quantiles or spread evenly over the range of the training data
- use_density If bin_continuous = FALSE should continuous data be sampled using a kernel density estimation. If not, continuous features are expected to follow a normal distribution.
LIME SETTINGS JUSTIFICATIONS
NOTE: many of these settings were obtained through trial and error, and/or based on the suggestions provided by LIME and its libraries. Since LIME’s results are not always the same, we used the settings that generated the most consistent and valid explanations during our experinments. Of course, we encourage others to try out other settings and experinment further. * Settings for “explainer” * bin_continuous = TRUE –> Makes explanation easier, and having this setting as true helped in obtaining more consistent and reliable results. * quantile_bins = FALSE –> setting this to TRUE generates an error b/c sameStrand and sameDownStrand don’t have enough variance for the algorithm to use quantile bining.
* use_density = TRUE –> with bin_continuous set to TRUE, this setting becomes irrelevant. In section 3.4, we showed that the features don’t follow a normal distribution. We tried setting bin_continuous = FALSE and use_density = TRUE, but this generated contradictory explanations that didn’t make sense. * n_bins = 10 –> The default value wasn’t giving us consistent results, so we bumped up the number to 10 and started getting seemingly good results
- Settings for “explain”
- n_labels = 1 –> we only have 1 label
- n_features = 7 –> we wanted our explanations to be based on all 7 features
- n_permutations = 2000 –> This number was set through trial and error. Having a small number of permutations does not generate consisten results, so a high number of permutations is recommended. 2000 just seemed to have yielded good results, so we decided to keep this value. If speed is a concern, lowering this number to 1000 should still yield good results.
- dist_fun = “gower” –> LIME’s default dist_fun, and since we have categorical features (sameStrand and sameDownStrand), according to LIME’s documentation, “gower” is the functions of choice when dealing with a mixture of numerical and categorical features
- Side Note: We tried a couple of runs of the script with euclidean distance, but it didn’t seem to have much of an impact. Additionally, the different types of settings, such as different gower powers, kernel widths, and distance functions, that can be tweaked in LIME means that a 100% thorough analysis of every possibility may be nearly impossible. For this reason, we tried to stick to the default settings as offered by LIME.
18. LIME PRE-SETUP
18.1 LIME compatability function
The original model was built using the randomForest library found in CRAN. However, even though LIME is supposed to be model agnostic, it’s current R implementation only supports certain models out of the box. To extend support of the LIME library to other models, the following functions need to be defined for each unsupported model type.
model_type.randomForest <- function(x,...) 'classification'
predict_model.randomForest <- function(x, newdata, type, ...) {
res <- predict(x, newdata = newdata, type = "prob")
switch(
type,
raw = data.frame(Response = ifelse(as.vector(res[,2]) > 0.5, "1", "0"), stringsAsFactors = FALSE),
prob = as.data.frame(res, check.names = F)
)
}18.2 Data to Explain
sampleData, sampleData_scaled, and sampleData_norm all contain the same features, and hence, we should be getting similar explanations in all of our results. We also apply LIME to the same instance 4 times, with the purpose of testing the validity and consistency of LIME’s explanations.
sampleData <- slt2data[1,] # A true sample
sampleData[c(2:4),] <- slt2data[1,]
sampleData[5,] <- slt2data[1986,] # A false sample
sampleData[c(6:8),] <- slt2data[1986,]
sampleDatasampleData_scaled <- slt2data_scaled[1,]
sampleData_scaled[c(2:4),] <- slt2data_scaled[1,]
sampleData_scaled[5,] <- slt2data_scaled[1986,]
sampleData_scaled[c(6:8),] <- slt2data_scaled[1986,]
sampleData_scaledsampleData_norm <- slt2data_norm[1,]
sampleData_norm[c(2:4),] <- slt2data_norm[1,]
sampleData_norm[5,] <- slt2data_norm[1986,]
sampleData_norm[c(6:8),] <- slt2data_norm[1986,]
sampleData_norm19. LIME with RandomForest Library
19.1 In original Model as is
While on occassion all results may look similar, we recommend running this particular section a few times, and you will notice significant changes, like features that may support the explanation in one plot, but contradict it in the next.
predict(origRF, sampleData[c(1,5),-c(8:9)], type = "prob") 0 1
Thr_leader 0.270 0.730
S_enterica_LT2_RAND_1598 0.965 0.035
attr(,"class")
[1] "matrix" "votes" lime_explainer_orig <- lime(as.data.frame(trainData[,c(1:7)]), origRF,
bin_continuous = TRUE, quantile_bins=FALSE, use_density = TRUE)
lime_explanations_orig <- explain(as.data.frame(sampleData[c(1:4),c(1:7)]), lime_explainer_orig, n_labels = 1, n_features = 7, n_permutations = 2000)
plot_features(lime_explanations_orig)
lime_explanations_orig <- explain(as.data.frame(sampleData[c(5:8),c(1:7)]), lime_explainer_orig, n_labels = 1, n_features = 7, n_permutations = 2000)skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeplot_features(lime_explanations_orig)
19.2 In original scaled Model
predict(origRF_scaled, sampleData_scaled[c(1,5),-c(8:9)], type = "prob") FALSE TRUE
Thr_leader 0.255 0.745
S_enterica_LT2_RAND_1598 0.950 0.050
attr(,"class")
[1] "matrix" "votes" lime_explainer_orig_scaled <- lime(as.data.frame(trainData_scaled[, c(1:7)]), origRF_scaled,
bin_continuous = TRUE, quantile_bins=FALSE,
use_density = FALSE, n_bins = 10)
norm_dist <- as.data.frame(lime_explainer_orig_scaled$bin_cuts)
lime_explainer_orig_scaled <- lime(as.data.frame(trainData_scaled[, c(1:7)]), origRF_scaled,
bin_continuous = TRUE, quantile_bins=FALSE,
use_density = TRUE, n_bins = 10)
not_norm_dist <- as.data.frame(lime_explainer_orig_scaled$bin_cuts)
# The correlations here show that the bins used by the explainer are the same regardless of what value is given to the use_density parameter.
cor(norm_dist[,"SS"],not_norm_dist[,"SS"])[1] 1cor(norm_dist[,"Pos10wrtsRNAStart"],not_norm_dist[,"Pos10wrtsRNAStart"])[1] 1cor(norm_dist[,"DistTerm"],not_norm_dist[,"DistTerm"])[1] 1cor(norm_dist[,"Distance"],not_norm_dist[,"Distance"])[1] 1cor(norm_dist[,"sameStrand"],not_norm_dist[,"sameStrand"])[1] 1cor(norm_dist[,"DownDistance"],not_norm_dist[,"DownDistance"])[1] 1cor(norm_dist[,"sameDownStrand"],not_norm_dist[,"sameDownStrand"])[1] 1lime_explainer_orig_scaled$bin_cuts # All the features seem to be evenly split into 10, ie, a density is not used for the bin splitting. $SS
[1] -4.3406919 -3.7652111 -3.1897304 -2.6142496 -2.0387689 -1.4632881 -0.8878074 -0.3123266 0.2631541 0.8386349 1.4141156
$Pos10wrtsRNAStart
[1] -1.9301214 -1.6047743 -1.2794272 -0.9540801 -0.6287330 -0.3033859 0.0219612 0.3473083 0.6726554 0.9980025 1.3233496
$DistTerm
[1] -1.7364372 -1.4902626 -1.2440880 -0.9979134 -0.7517387 -0.5055641 -0.2593895 -0.0132149 0.2329597 0.4791343 0.7253090
$Distance
[1] -14.828223 -13.324592 -11.820961 -10.317330 -8.813699 -7.310068 -5.806437 -4.302807 -2.799176 -1.295545 0.208086
$sameStrand
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
$DownDistance
[1] -0.1934291 1.7162357 3.6259006 5.5355654 7.4452302 9.3548950 11.2645599 13.1742247 15.0838895 16.9935543 18.9032192
$sameDownStrand
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0lime_explanations_orig_scaled <- explain(as.data.frame(sampleData_scaled[c(1:4),c(1:7)]),
lime_explainer_orig_scaled, n_labels = 1,
n_features = 7, n_permutations = 2000
)
plot_features(lime_explanations_orig_scaled)
lime_explanations_orig_scaled <- explain(as.data.frame(sampleData_scaled[c(5:8),c(1:7)]),
lime_explainer_orig_scaled, n_labels = 1,
n_features = 7, n_permutations = 2000
)skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeplot_features(lime_explanations_orig_scaled)
19.3 In original normalized Model
predict(origRF_norm, sampleData_norm[c(1,5),-c(8:9)], type = "prob") FALSE TRUE
Thr_leader 0.2575 0.7425
S_enterica_LT2_RAND_1598 0.9500 0.0500
attr(,"class")
[1] "matrix" "votes" lime_explainer_orig_norm <- lime(as.data.frame(trainData_norm[, c(1:7)]), origRF_norm,
bin_continuous = TRUE, quantile_bins=FALSE,
use_density = TRUE, n_bins = 10)
lime_explainer_orig_norm$bin_cuts # all bins are split evenly, from 0 to 1, in increments of 0.1, as would be expected$SS
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
$Pos10wrtsRNAStart
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
$DistTerm
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
$Distance
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
$sameStrand
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
$DownDistance
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
$sameDownStrand
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0lime_explanations_orig_norm <- explain(as.data.frame(sampleData_norm[c(1:4),c(1:7)]),
lime_explainer_orig_norm, n_labels = 1,
n_features = 7, n_permutations = 2000
)
plot_features(lime_explanations_orig_norm)
lime_explanations_orig_norm <- explain(as.data.frame(sampleData_norm[c(5:8),c(1:7)]),
lime_explainer_orig_norm, n_labels = 1,
n_features = 7, n_permutations = 2000
)skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeplot_features(lime_explanations_orig_norm)
20. LIME in Orig RF Conclusions
We tried applying LIME to the Orig RF(in part 15) hoping it would work like a plug-and-play application. To our surprise, you can’t get LIME to properly work with the “Original” RF model because you must normalize, or scale the data. LIME generates random permutations to the features in order to create a linear model, but if the features have different scales and values, it is plausible that features with higher scales will simply outweight the other features, even if they’re significantly less important, and hence create skewed models and explanations. To test this hypothesis, we created a GLM model and reviewed its R^2 value, which was suprisingly low, and hence, finished convincing us that the linear models generated by LIME with the OrigRF were inadequate. Additionally, since LIME randomizes the permutation points, based on our experinments, it seems that LIME is using its own randomization algorith (ie. we couldn’t set a seed for it), and hence, the exact results we obtained with the OrigRF may not be completely repeatable. However, for the particular case in part 15, that may be irrelevant as the point we are trying to prove there is that LIME’s explanations when the data is not scaled or normalized are inconsistent, and hence cannot be trusted.
For each LIME explanation plot, since we’re having LIME explain the same instance, all 4 plots should be identical to each other. However, with the OrigRF model, there’s usually at least 1 plot that ranks the feature contributions differently, or that may have a feature “Supporting” in one plot, but “Contradicting” in the next. Due to LIME’s nature, obtaining slightly different results was expected, but not to a degree that would make the explanations invalid.
By scaling, or normalizing, the data, we were able to obtain more consistent results (ie. features that support or contradict a prediction, behave the same in all the corresponding 4 plots, although their importance ranking may be different), and in most cases, the rankings of feature contributions as plotted by LIME actually remained the same. LIME behaves properly if the data is scaled and normalized.
An additional interesting take-away, is that sameStrand and sameDownStrand are considered the least important features, and seem to become completely irrelevant for predictions where the instance is FALSE as LIME simply exclude this feature .
21. LIME for the H2O RF models
Initial struggles with getting LIME set up with the original random forest lead us to believe that since RandomForest wasn’t a supported method by default in the Lime Library, that maybe using a supported method would help us overcome these problems. During our investigation, we discovered that H2O offered additional IMs (PDPs and SHAP), and that their algorithms were natively supported by the Lime library. Thinking we could kill 3 birds (test 3 IMs) with one stone (one package), we proceeded to create the equivalent proxy models in H2O and run the corresponding test. The first H2O RF model was created prior to realizing we needed to have normalized and scaled data sets, as these data preprocessing techniques are normally not necessary for random forests. Further investigations and testing allowed us to discover that normalizing and scaling the data allowed LIME to generate reasonable explanations with the OrigRF model. Eventually we decided to create 5 models in total, 2 with the original RandomForest library, and 3 with the H2O library. Even though the performance metrics for the scaled and normalized H2O RF models did not match those of the OrigRF, we decided to still include them in the following sections.
21.1 H2O RF Model
h2o.predict(object = rfh2o, newdata = as.h2o(sampleData[c(1,8),c(1:7)]) )
[2 rows x 3 columns] lime_explainer_rfh2o <- lime( as.data.frame( trainData[,c(1:7)] ), rfh2o,
bin_continuous = TRUE, quantile_bins = FALSE,
n_bins = 10
)
lime_explanations_rfh2o <- explain( as.data.frame(sampleData[c(1:4),c(1:7)] ),lime_explainer_rfh2o,
n_labels = 1, n_features = 7,
n_permutations = 2000 )
plot_features(lime_explanations_rfh2o) 
lime_explanations_rfh2o <- explain( as.data.frame(sampleData[c(5:8),c(1:7)] ),lime_explainer_rfh2o,
n_labels = 1, n_features = 7,
n_permutations = 2000 )skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeplot_features(lime_explanations_rfh2o) 
21.2 In H2O RF scaled Model
h2o.predict(object = rfh2o_scaled, newdata = as.h2o(sampleData_scaled[c(1,8),c(1:7)]))
[2 rows x 3 columns] lime_explainer_rfh2o_scaled <- lime( as.data.frame( trainData_scaled[,c(1:7)] ), rfh2o_scaled,
bin_continuous = TRUE, quantile_bins = FALSE)
lime_explanations_rfh2o_scaled <- explain( as.data.frame(sampleData_scaled[c(1:4),c(1:7)]),
lime_explainer_rfh2o_scaled, n_labels = 1,
n_features = 7, n_permutations = 2000, dist_fun = "gower" )
plot_features(lime_explanations_rfh2o_scaled)
lime_explanations_rfh2o_scaled <- explain( as.data.frame(sampleData_scaled[c(5:8),c(1:7)]),
lime_explainer_rfh2o_scaled, n_labels = 1,
n_features = 7, n_permutations = 2000, dist_fun = "gower" )skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeplot_features(lime_explanations_rfh2o_scaled)
21.3 LIME to H2O RF normalized Model
h2o.predict(object = rfh2o_norm, newdata = as.h2o(sampleData_norm[c(1,8),c(1:7)]) )
[2 rows x 3 columns] lime_explainer_rfh2o_norm <- lime( as.data.frame( trainData_norm[,c(1:7)] ), rfh2o_norm,
bin_continuous = TRUE, quantile_bins = FALSE, use_density = TRUE)
lime_explanations_rfh2o_norm <- explain( as.data.frame(sampleData_norm[c(1:4),c(1:7)] ),
lime_explainer_rfh2o_norm, n_labels = 1,
n_features = 7, n_permutations = 2000, dist_fun = "gower" )
plot_features(lime_explanations_rfh2o_norm)
lime_explanations_rfh2o_norm <- explain( as.data.frame(sampleData_norm[c(5:8),c(1:7)] ),
lime_explainer_rfh2o_norm, n_labels = 1,
n_features = 7, n_permutations = 2000, dist_fun = "gower" )skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeplot_features(lime_explanations_rfh2o_norm)
21.4 Applying LIME to GLM Model (BONUS)
lime_explainer_glmh2o <- lime( as.data.frame( trainData[,c(1:7)] ),glmh2o,
bin_continuous = TRUE, quantile_bins = FALSE,
use_density = FALSE, n_bins = 10 )
lime_explanations_glmh2o <- explain( as.data.frame(sampleData[c(1:4),c(1:7)] ), lime_explainer_glmh2o,
n_labels = 1, n_features = 7,
n_permutations = 2000, dist_fun = "gower" )
plot_features(lime_explanations_glmh2o)
lime_explanations_glmh2o <- explain( as.data.frame(sampleData[c(5:8),c(1:7)] ), lime_explainer_glmh2o,
n_labels = 1, n_features = 7,
n_permutations = 2000, dist_fun = "gower" )skipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeskipping variable with zero or non-finite rangeplot_features(lime_explanations_glmh2o)
25. LIME CONCLUSIONS
A known problem with LIME is that explanations may be inconsistent from one instance to the next, even when the instances are very similar to each. With this in mind, each instance we tested was tested at least 4 times (we manually ran this section over and over again) , and with a high enough number of permutations, we managed to get somewhat consistent results. However, some of the results obtained from LIME were heavily flawed as sometimes it would yield explanations that contradicted the result. For some reason, when explaining false instances, LIME would tend to leave out sameStrand and sameDownStrand from the explanation. Additionally, a GLM was also trained to attempt to measure the variance of the data, but the results from the GLM yielded an R^2 value between 0.2 and 0.3, with a max recall of 0.07. In other words, the GLM model was only predicting false instances, and since LIME uses linear models for its explanations, based on our observations, it is safe to conclude that LIME was doing the same. Whenever a TRUE instance was analyze, LIME’s explanations would say that most factors contradict the prediction, which is obviously counter intuitive and useless.
During the making of this project, several problems were encountered while trying to apply LIME to the sRNA RF model. LIME attempts to explain a complex model’s behavior in a small region corresponding to a particular instance of interest by applying a linear model. However, in this particular use case, the mixture of categorical and numerical values, plus the differences in ranges within each feature resulted in various experinments without consistent results. For this particular use case, LIME was implement in models trained with the original training data, scaled training data, and normalized training data. Several attempts were also made at modifying the arguments for the lime() function, such as all the possible permutations between bin_continuous, quantile_bins and use_density. In the explain function, the number of permutations was increased from 10 to 2000 and the Euclidean and Gower distance functions were tested with kernel_widths ranging from 0.001 to 5 because we were not getting consistent
F) SHAP Values
SHAP_H2O1 <- h2o.predict_contributions(rfh2o, as.h2o(sampleData[,c(1:7)]))
head(SHAP_H2O1,10)G) PDP
trainData[,c(1:9)]
[652 rows x 9 columns] LS0tCnRpdGxlOiAiSW50ZXJwcmV0YWJpbGl0eSBNb2RlbHMgaW4gdGhlIFJhbmRvbSBGb3Jlc3Qgc1JOQSIKb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6IGRlZmF1bHQKICBwZGZfZG9jdW1lbnQ6IGRlZmF1bHQKLS0tCiMgVGFibGUgb2YgQ29udGVudHMKMS4gUFJFLVNFVFVQID0gV2UgYXJlIHNpbXBseSBnZXR0aW5nIHRoZSBlbnZpcm9ubWVudCBhbmQgZGF0YSByZWFkeQoxLiBUUkFJTiBORVcgSDJPIE1PREVMID0gVG8gbWFrZSB1c2Ugb2YgSDJPJ3MgbGlicmFyaWVzLCB3ZSBuZWVkIHRvIGNyZWF0ZSB0aGUgbW9kZWxzIGluIEgyTyAoaWUuIG91ciBvcmlnaW5hbCBSRiBtb2RlbCBjYW5ub3QgYmUgZGlyZWN0bHkgY29udmVydGVkIHRvIGFuIGgybyBtb2RlbCkKMS4gQ09NUEFSRSBNT0RFTCdTIFBSRURJQ1RJT05TID0gV2UgaGF2ZSB0aGUgT3JpZ1JGIGFuZCBhbGwgdGhlIG90aGVyIG5ldyBtb2RlbHMgZ2VuZXJhdGUgcHJlZGljdGlvbnMgb24gdGhlIHRlc3RpbmcgZGF0YQoxLiBDT01QQVJFIE1PREVMJ1MgTUVUUklDUyA9IEEgdHJ1ZSB2YWxpZGF0aW9uIG9mIHRoZSBtb2RlbHMnIHBlcmZvcm1hbmNlLCBhbmQgYSBjb3JyZWN0IHdheSBvZiB2YWxpZGF0aW5nIGlmIG91ciBuZXcgbW9kZWxzIG1ha2UgZ29vZCBwcm94aWVzIGZvciB0aGUgb3JpZ2luYWwgbW9kZWwKMS4gUkFORE9NIEZPUkVTVCBFWFBMQUlORVIgKG9yIFBEUHMpID0gV2UgYXBwbGllZCBSRkUgdG8gb3VyIG9yaWdSRiBtb2RlbAoxLiBMSU1FID0gSGVyZSB3ZSBhY3R1YWxseSBiZWdpbiB0byBhcHBseSBMSU1FIHRvIG91ciBSRiBtb2RlbHMKMS4gU0hBUCBWYWx1ZXMgPSBXZSB1c2UgdGhlIEgyTyBsaWJyYXJ5IHRvIGdlbmVyYXRlIHRoZSBTSEFQIHZhbHVlcyBmb3Igb3VyIEgyTyBSRiBtb2RlbAoxLiBQRFBzID0gU2ltaWxhciB0byB0aGUgcHJldmlvdXMgc2VjdGlvbiwgd2UgdXNlIHRoZSBIMk8gbGlicmFyeSB0byBnZW5lcmF0ZSB0aGUgUERQcyBmb3IgdGhlIEgyTyBSRiBtb2RlbAoKIyMgSW50cm9kdWN0aW9uClRoZSBnb2FsIG9mIHRoaXMgcHJvamVjdCB3YXMgdG8gYXBwbHkgaW50ZXJwcmV0YWJpbGl0eSBtb2RlbHMgdG8gdGhlIHJhbmRvbSBmb3Jlc3QgY2xhc3NpZmllciBpbiB0aGUgUGVlckogYXJ0aWNsZSBbIlByaW9yaXRpemluZyBib25hIGZpZGUgYmFjdGVyaWFsIHNtYWxsIFJOQXMgd2l0aCBtYWNoaW5lIGxlYXJuaW5nIGNsYXNzaWZpZXJzIl0oaHR0cHM6Ly9wZWVyai5jb20vYXJ0aWNsZXMvNjMwNC8pIHdpdGggdGhlIHB1cnBvc2VzIG9mIGdhaW5pbmcgZGVlcGVyIGluc2lnaHRzIGludG8gdGhlIHByaW9yaXRpemF0aW9uIG9mIGJvbmEgZmlkZSBiYWN0ZXJpYWwgc1JOQXMuIAoKCiMjIFNvbWUga2V5IFRlcm1zCiogT3JpZ1JGL09yaWcgUkYgPSBPcmlnaW5hbCByYW5kb20gZm9yZXN0IGFzIGZvdW5kIGluIFtsaW5rXShodHRwczovL2dpdGh1Yi5jb20vQmlvaW5mb3JtYXRpY3NMYWJBdE1VTi9zUk5BUmFua2luZykKKiBPcmlnUkZfW3NjYWxlZC9ub3JtYWxpemVkXSA9IHJlZmVycyB0byBhbnkgbW9kZWwgdGhhdCB3YXMgdHJhaW5lZCBpbiB0aGUgc2FtZSBtYW5uZXIoc2FtZSBsaWJyYXJpZXMgYW5kIHNldHRpbmdzKSBhcyBpbiB0aGUgT3JpZ2luYWwgcmFuZG9tIGZvcmVzdCBtb2RlbCwgYnV0IHVzaW5nIHNjYWxlZCBvciBub3JtYWxpemVkIGRhdGEuIAogICogIEJ5IHNjYWxpbmcgdGhlIGRhdGEgd2UgbWVhbiBzY2FsaW5nIGFuZCBjZW50ZXJpbmcKICAqICBCeSBub3JtYWxpemluZyB3ZSBtZWFuIGZpdCB0aGUgdmFsdWVzIGJldHdlZW4gMCBhbmQgMQoqIFJGRSA9IFJhbmRvbSBGb3Jlc3QgRXhwbGFpbmVyCiogUkYgPSBSYW5kb20gRm9yZXN0CiogSU0gPSBJbnRlcnByZXRhYmlsaXR5IE1vZGVsCiogIlNTIiA9IHRoZSBmcmVlIGVuZXJneSBvZiB0aGUgcHJlZGljdGVkIHNlY29uZGFyeSBzdHJ1Y3R1cmUgb2YgdGhlIHNSTkEgLSBtb3N0bHkgbmVnYXRpdmUgdmFsdWVzCiogIlBvczEwd3J0c1JOQVN0YXJ0IiA9IGRpc3RhbmNlIHRvIHRoZWlyIGNsb3Nlc3QgcHJlZGljdGVkIHByb21vdGVyIHNpdGUKKiAiRGlzdFRlcm0iID0gZGlzdGFuY2UgdG8gdGhlaXIgY2xvc2VzdCBwcmVkaWN0ZWQgUmhvLWluZGVwZW5kZW50IHRlcm1pbmF0b3IgCiogIkRpc3RhbmNlIiA9IGRpc3RhbmNlIHRvIHRoZSBjbG9zZXN0IHJlYWRpbmcgZnJhbWUgb24gdGhlIExFRlQoInVwc3RyZWFtIikgc2lkZQoqICJzYW1lU3RyYW5kIiA9IGJvb2xlYW4sIGlmIHRyYW5zY3JpcHRpb24gaXMgZ29pbmcgaW4gdGhlIHNhbWUgZGlyZWN0aW9uIGFzIHRoZSBPUkYoTGVmdCBPcGVuIFJlYWRpbmcgRnJhbWUpCiAgKiAgT1JGID0gZ2Vub21pYyBzZXF1ZW5jZSB0aGF0J3Mgc3VwcG9zZWQgdG8gY29kZSBmb3IgYSBwcm90ZWluCiogIkRvd25EaXN0YW5jZSIgID0gZGlzdGFuY2UgdG8gdGhlIGNsb3Nlc3QgcmVhZGluZyBmcmFtZSBvbiB0aGUgUklHSFQoImRvd25zdHJlYW0iKSBzaWRlCiogInNhbWVEb3duU3RyYW5kIiA9ICBib29sZWFuLCBpZiB0cmFuc2NyaXB0aW9uIGlzIGdvaW5nIGluIHRoZSBzYW1lIGRpcmVjdGlvbiBhcyB0aGUgUk9SRihSaWdodCBPcGVuIFJlYWRpbmcgRnJhbWUpCgoqKlNpZGUgbm90ZToqKiogV2hlbiB1c2VkIHRoZSB3b3JkIG9yaWdpbmFsIHRvIHJlZmVyIHRvIHRoZSBSRiBtb2RlbCBjcmVhdGVkIGluIHRoZSBiYXNlIGFydGljbGUsIG9yIHRvIHJlZmVyIHRvIG1vZGVscyB0aGF0IHdlcmUgY3JlYXRlZCh0cmFpbmVkKSBpbiB0aGUgc2FtZSB3YXkgYXMgdGhlIGJhc2UgcmFuZG9tIGZvcmVzdCBtb2RlbC4gCgojIEkpIFBSRS1TRVRVUApUaGlzIHNlY3Rpb24gaXMgYWxsIGFib3V0IGdldHRpbmcgdGhlIGxpYnJhcmllcyBhbmQgZGF0YSBuZWVkZWQgdG8gcnVuIHRoaXMgUiBub3RlYm9vay4gV2Ugd2lsbCBwcmVwcm9jZXNzIHRoZSBkYXRhIGFuZCBsb2FkIHRoZSBvcmlnaW5hbCBtb2RlbC4gV2Ugc3RhcnQgd2l0aCBzb21lICoqT1BUSU9OQUwqKiBtZW1vcnkgY2xlYW5pbmcgKHVzZWZ1bCBpZiB5b3Ugd2FudCB0byBydW4gdGhpcyBmcm9tIFIgU3R1ZGlvIGFuZCB3aXRoIGEgY2xlYW4gZW52aXJvbm1lbnQpCmBgYHtyfQojIFRoaXMgc2VjdGlvbiBpcyBqdXN0IGFib3V0IGdldHRpbmcgdGhlIGVudmlyb25tZW50IHJlYWR5IGFuZCBjbGVhbmVkIHVwLiBTaW5jZSB0aGlzIHByb2plY3Qgd2FzIG9yaWdpbmFsbHkgd3JpdHRlbiBhcyBhIHJlZ3VsYXIgUiBzY3JpcHQsIGFuZCB3b3JrZWQgZnJvbSB3aXRoaW4gUiBTdHVkaW8sIGl0IHdhcyBqdXN0IHVzZWZ1bGwgaW4gcmVzZXR0aW5nIGFuZCBjbGVhbmluZyB1cCB0aGUgdGhlIGVudmlyb25tZW50LCBhbmQgc28gdGhpcyBibG9jayBvZiBjb2RlIGlzIG9wdGlvbmFsCmNsYyA8LSBmdW5jdGlvbigpIGNhdCgiXDAxNCIpIDsgY2xjKCkKcmVtb3ZlKGxpc3QgPSBscygpKQpnYygpCmBgYAoKIyMgMS4gSW5zdGFsbCByZXF1aXJlZCBsaWJyYXJpZXMgCgpUaGUgZm9sbG93aW5nIGNvZGUgaXMgYmFzZWQgb24gdGhlIGgybyBkb2N1bWVudGF0aW9uIHNpdGUsIGFzIGl0J3MgdGhlaXIgcmVjb21tZW5kZWQgd2F5IGZvciBkb3dubG9hZGluZyBhbmQgaW5zdGFsbGluZyBoMm8gZm9yIFIsIHdpdGggb25seSB0aGUgZmlyc3QgaWYgYmVpbmcgYWRkZWQgYXMgdG8gcHJldmVudCBhY2NpZGVudGFsIHJlaW5zdGFsbGF0aW9ucy4gSGVyZSBpcyB0aGUgW2xpbmtdKGh0dHA6Ly9oMm8tcmVsZWFzZS5zMy5hbWF6b25hd3MuY29tL2gyby9yZWwtemFocmFkbmlrLzQvaW5kZXguaHRtbCkgZm9yIHRob3NlIGludGVyZXN0ZWQgaW4gaXQuICAKCmBgYHtyfQoKaWYgKCEgKCJoMm8iICVpbiUgcm93bmFtZXMoaW5zdGFsbGVkLnBhY2thZ2VzKCkpKSkgeyAKICAKICAgICMgVGhlIGZvbGxvd2luZyB0d28gY29tbWFuZHMgcmVtb3ZlIGFueSBwcmV2aW91c2x5IGluc3RhbGxlZCBIMk8gcGFja2FnZXMgZm9yIFIuCiAgICBpZiAoInBhY2thZ2U6aDJvIiAlaW4lIHNlYXJjaCgpKSB7IGRldGFjaCgicGFja2FnZTpoMm8iLCB1bmxvYWQ9VFJVRSkgfQogICAgaWYgKCJoMm8iICVpbiUgcm93bmFtZXMoaW5zdGFsbGVkLnBhY2thZ2VzKCkpKSB7IHJlbW92ZS5wYWNrYWdlcygiaDJvIikgfQogICAgIyBOZXh0LCB3ZSBkb3dubG9hZCBwYWNrYWdlcyB0aGF0IEgyTyBkZXBlbmRzIG9uLgogICAgcGtncyA8LSBjKCJSQ3VybCIsImpzb25saXRlIikKICAgIGZvciAocGtnIGluIHBrZ3MpIHsKICAgICAgaWYgKCEgKHBrZyAlaW4lIHJvd25hbWVzKGluc3RhbGxlZC5wYWNrYWdlcygpKSkpIHsgaW5zdGFsbC5wYWNrYWdlcyhwa2cpIH0KICAgIH0KICAgICMgTm93IHdlIGRvd25sb2FkLCBpbnN0YWxsIGFuZCBpbml0aWFsaXplIHRoZSBIMk8gcGFja2FnZSBmb3IgUi4KICAgIGluc3RhbGwucGFja2FnZXMoImgybyIsIHR5cGU9InNvdXJjZSIsIHJlcG9zPSJodHRwOi8vaDJvLXJlbGVhc2UuczMuYW1hem9uYXdzLmNvbS9oMm8vcmVsLXphaHJhZG5pay80L1IiKQp9ICAKYGBgCgpUaGUgcmVzdCBvZiB0aGUgbGlicmFyaWVzIGNhbiBiZSBpbnN0YWxsZWQgaW4gdGhlIHNhbWUgd2F5IGFzIHdpdGggYW55IG90aGVyIGxpYnJhcnkuCmBgYHtyfQpwYWNrYWdlcyA8LSBjKCJsaW1lIiwgIlJPQ1IiLCAiUFJST0MiLCAicmFuZG9tRm9yZXN0IiwgInJhbmRvbUZvcmVzdEV4cGxhaW5lciIsICJwbmciKQpmb3IgKHBhY2thZ2UgaW4gcGFja2FnZXMpIHsKICBpZiAoISAocGFja2FnZSAlaW4lIHJvd25hbWVzKGluc3RhbGxlZC5wYWNrYWdlcygpKSkpIHsgaW5zdGFsbC5wYWNrYWdlcyhwYWNrYWdlKSB9Cn0KCmBgYAoKIyMgMi4gTG9hZCB0aGUgbGlicmFyaWVzIAoKYGBge3J9CmxpYnJhcnkoImgybyIpICAgICAgICAgICMgTUwgbW9kZWwgYnVpbGRpbmcKbGlicmFyeSgiUk9DUiIpICAgICAgICAgIyBNTCBldmFsdWF0aW9uCmxpYnJhcnkoIlBSUk9DIikgICAgICAgICMgTUwgZXZhbHVhdGlvbgpsaWJyYXJ5KCJyYW5kb21Gb3Jlc3QiKSAjIE1MIG1vZGVsIGJ1aWxkaW5nCmxpYnJhcnkoInJhbmRvbUZvcmVzdEV4cGxhaW5lciIpICMgTUwgR2xvYmFsIGludGVycHJldGF0aW9uCmxpYnJhcnkoImxpbWUiKSAgICAgICAgICMgTUwgbG9jYWwgaW50ZXJwcmV0YXRpb24KbGlicmFyeSgicG5nIikKYGBgCgojIyAzLiBMb2FkIGFuZCBwcmVwcm9jZXNzIHRoZSBkYXRhIAoKIyMjIDMuMSBMb2FkIHRoZSBNb2RlbCdzIFRyYWluaW5nIERhdGEgCmBgYHtyfQp0cmFpbkRhdGFTZXQgPC0gcmVhZC5jc3YoIi4vRGF0YVNldHMvY29tYmluZWREYXRhLmNzdiIsIGhlYWRlciA9IFRSVUUpCnRyYWluRGF0YVNldFssIkNsYXNzIl0gPC0gYXMubG9naWNhbCh0cmFpbkRhdGFTZXRbLCJDbGFzcyJdKSAjIGd1YXJhbnRlZXMgdGhhdCBvdXIgaDJvIG1vZGVsIHRyYWlucyB0aGUgcmFuZG9tIGZvcmVzdCAoUkYpIGFzIGEgY2xhc3NpZmljYXRpb24gdHJlZSBhbmQgbm90IGEgcmVncmVzc2lvbiB0cmVlCmBgYAoKIyMjIDMuMiBTY2FsZSBhbmQgQ2VudGVyIHRoZSBUcmFpbmluZyBEYXRhIAoKYGBge3J9CmRhdGFTZXRUcmFpbl9zY2FsZWQgPC0gc2NhbGUodHJhaW5EYXRhU2V0WywtYyg1LDcsOCw5KV0sIGNlbnRlciA9IFRSVUUsIHNjYWxlID0gVFJVRSkKZGF0YVNldFRyYWluX3NjYWxlZCA8LSBjYmluZCggZGF0YVNldFRyYWluX3NjYWxlZFssKDE6NCldLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhU2V0Wyw1XSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZGF0YVNldFRyYWluX3NjYWxlZFssNV0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YVNldFssYyg3LDgsOSldKQpjb2xuYW1lcyhkYXRhU2V0VHJhaW5fc2NhbGVkKSA8LSBjKCJTUyIsICJQb3MxMHdydHNSTkFTdGFydCIsICJEaXN0VGVybSIsICJEaXN0YW5jZSIsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJzYW1lU3RyYW5kIiwgIkRvd25EaXN0YW5jZSIsICJzYW1lRG93blN0cmFuZCIsICJJRCIsICJDbGFzcyIpCmRhdGFTZXRUcmFpbl9zY2FsZWRbLCJDbGFzcyJdIDwtIGFzLmxvZ2ljYWwoZGF0YVNldFRyYWluX3NjYWxlZFssIkNsYXNzIl0pICMgSnVzdCBtYWtpbmcgc3VyZSBDbGFzcyBpcyB0YWtlbiBhIGxvZ2ljYWwgdmFsdWUKcmJpbmQoIGhlYWQoZGF0YVNldFRyYWluX3NjYWxlZCw1KSwgdGFpbChkYXRhU2V0VHJhaW5fc2NhbGVkLDUpICkKCmBgYAoKIyMjIDMuMyBOb3JtYWxpemUgdGhlIFRyYWluaW5nIERhdGEKYGBge3J9Cm5vcm1hbGl6ZSA8LSBmdW5jdGlvbih4KSB7CiAgcmV0dXJuICgoeCAtIG1pbih4KSkgLyAobWF4KHgpIC0gbWluKHgpKSkKfQoKZGF0YVNldFRyYWluX25vcm0gPC0gdHJhaW5EYXRhU2V0CmRhdGFTZXRUcmFpbl9ub3JtJFNTIDwtIG5vcm1hbGl6ZShkYXRhU2V0VHJhaW5fbm9ybSRTUykKZGF0YVNldFRyYWluX25vcm0kUG9zMTB3cnRzUk5BU3RhcnQgPC0gbm9ybWFsaXplKGRhdGFTZXRUcmFpbl9ub3JtJFBvczEwd3J0c1JOQVN0YXJ0KQpkYXRhU2V0VHJhaW5fbm9ybSREaXN0VGVybSA8LSBub3JtYWxpemUoZGF0YVNldFRyYWluX25vcm0kRGlzdFRlcm0pCmRhdGFTZXRUcmFpbl9ub3JtJERpc3RhbmNlIDwtIG5vcm1hbGl6ZShkYXRhU2V0VHJhaW5fbm9ybSREaXN0YW5jZSkKZGF0YVNldFRyYWluX25vcm0kRG93bkRpc3RhbmNlIDwtIG5vcm1hbGl6ZShkYXRhU2V0VHJhaW5fbm9ybSREb3duRGlzdGFuY2UpCmRhdGFTZXRUcmFpbl9ub3JtWywiQ2xhc3MiXSA8LSBhcy5sb2dpY2FsKGRhdGFTZXRUcmFpbl9ub3JtWywiQ2xhc3MiXSkgIyBKdXN0IG1ha2luZyBzdXJlIENsYXNzIGlzIHRha2VuIGEgbG9naWNhbCB2YWx1ZQpyYmluZCggaGVhZChkYXRhU2V0VHJhaW5fbm9ybSw1KSwgdGFpbChkYXRhU2V0VHJhaW5fbm9ybSw1KSApIApgYGAKCiMjIyAzLjQgQ2hlY2sgZm9yIG5vcm1hbGl0eSBpbiB0aGUgZGF0YQpUaGVyZSBhcmUgY2VydGFpbiBwYXJhbWV0ZXIoKSBpbiB0aGUgTElNRSBmdW5jdGlvbnMgdGhhdCBhcmUgcmVjb21tZW5kZWQgYmFzZWQgb24gdGhlIGRpc3RyaWJ1dGlvbiBvZiB0aGUgZGF0YS4gQmFzZWQgb24gdGhlc2UgcGFyZW1ldGVycywgd2UgZGVjaWRlZCB0byBjaGVjayBmb3Igbm9ybWFsaXR5IGluIHRoZSBkYXRhCgpgYGB7cn0KZGF0YVNldFRyYWluWCA8LSB0cmFpbkRhdGFTZXRbLC0oODo5KV0KZGF0YVNldFRyYWluWSA8LSB0cmFpbkRhdGFTZXRbLCg4OjkpXSAgCgpoaXN0KHggPSBhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDFdKSwgbWFpbiA9ICJIaXN0b2dyYW0gb2YgU1MiKQpxcW5vcm0oYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWywxXSkpCnFxbGluZShhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDFdKSwgY29sID0gInJlZCIsIGx3ZCA9IDIpCgpoaXN0KHggPSBhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDJdKSwgbWFpbiA9ICJIaXN0b2dyYW0gb2YgUG9zMTB3cnRzUk5BU3RhcnQiKQpxcW5vcm0oYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWywyXSkpCnFxbGluZShhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDJdKSwgY29sID0gInJlZCIsIGx3ZCA9IDIpCgpoaXN0KHggPSBhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDNdKSwgbWFpbiA9ICJIaXN0b2dyYW0gb2YgRGlzdFRlcm0iKQpxcW5vcm0oYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWywzXSkpCnFxbGluZShhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDNdKSwgY29sID0gInJlZCIsIGx3ZCA9IDIpCgpoaXN0KHggPSBhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDRdKSwgbWFpbiA9ICJIaXN0b2dyYW0gb2YgRGlzdGFuY2UiKQpxcW5vcm0oYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw0XSkpCnFxbGluZShhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDRdKSwgY29sID0gInJlZCIsIGx3ZCA9IDIpCgpoaXN0KHggPSBhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDVdKSwgbWFpbiA9ICJIaXN0b2dyYW0gb2Ygc2FtZVN0cmFuZCIpCnFxbm9ybShhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDVdKSkKcXFsaW5lKGFzLm51bWVyaWMoZGF0YVNldFRyYWluWFssNV0pLCBjb2wgPSAicmVkIiwgbHdkID0gMikKCmhpc3QoeCA9IGFzLm51bWVyaWMoZGF0YVNldFRyYWluWFssNl0pLCBtYWluID0gIkhpc3RvZ3JhbSBvZiBEb3duRGlzdGFuY2UiKQpxcW5vcm0oYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw2XSkpCnFxbGluZShhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDZdKSwgY29sID0gInJlZCIsIGx3ZCA9IDIpCgpoaXN0KHggPSBhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDddKSwgbWFpbiA9ICJIaXN0b2dyYW0gb2Ygc2FtZURvd25TdHJhbmQiKQpxcW5vcm0oYXMubnVtZXJpYyhkYXRhU2V0VHJhaW5YWyw3XSkpCnFxbGluZShhcy5udW1lcmljKGRhdGFTZXRUcmFpblhbLDddKSwgY29sID0gInJlZCIsIGx3ZCA9IDIpCmBgYAoKIyMjIDMuNSBMb2FkIHRlc3RpbmcgRGF0YXNldHMKTG9hZCBPcmlnaW5hbCBUZXN0aW5nIERhdGEgU2V0cwoKYGBge3J9CnNsdDJkYXRhUG9zIDwtIHJlYWQuY3N2KCIuL0RhdGFTZXRzL1NMVDJfUG9zaXRpdmVzLnRzdiIsIHNlcCA9ICJcdCIsIGhlYWRlciA9IFRSVUUpCnNsdDJkYXRhUG9zJENsYXNzIDwtIHJlcCgxLG5yb3coc2x0MmRhdGFQb3MpKQpzbHQyZGF0YU5lZyA8LSByZWFkLmNzdigiLi9EYXRhU2V0cy9TTFQyX05lZ2F0aXZlcy50c3YiLCBzZXAgPSAiXHQiLCBoZWFkZXIgPSBUUlVFKQpzbHQyZGF0YU5lZyRDbGFzcyA8LSByZXAoMCxucm93KHNsdDJkYXRhTmVnKSkKc2x0MmRhdGEgPC0gcmJpbmQoc2x0MmRhdGFQb3Msc2x0MmRhdGFOZWcpCgpsdWRhdGFQb3MgPC0gcmVhZC5jc3YoIi4vRGF0YVNldHMvTHVfUG9zaXRpdmVzLnRzdiIsIHNlcCA9ICJcdCIsIGhlYWRlciA9IFRSVUUpCmx1ZGF0YVBvcyRDbGFzcyA8LSByZXAoMSxucm93KGx1ZGF0YVBvcykpCmx1ZGF0YU5lZyA8LSByZWFkLmNzdigiLi9EYXRhU2V0cy9MdV9OZWdhdGl2ZXMudHN2Iiwgc2VwID0gIlx0IiwgaGVhZGVyID0gVFJVRSkKbHVkYXRhTmVnJENsYXNzIDwtIHJlcCgwLG5yb3cobHVkYXRhTmVnKSkKbHVkYXRhIDwtIHJiaW5kKGx1ZGF0YVBvcyxsdWRhdGFOZWcpCmBgYAoKIyMjIDMuNiBTY2FsZSB0ZXN0aW5nIERhdGFzZXRzCkV2ZW4gdGhvdWdoIHRoZSAqc2NhbGUgZnVuY3Rpb24qIGNhbiBzY2FsZSBhbmQgY2VudGVyIHRoZSBkYXRhIGZvciB1cywgd2UgbmVlZCB0byBzY2FsZSB0aGUgdGVzdGluZyBkYXRhIHNldHMgdXNpbmcgdGhlIHRyYWluaW5nIGRhdGEncyBzdGFuZGFyZCBkZXZpYXRpb24gYW5kIG1lYW5zIHRvIG9idGFpbiBhZGVxdWF0ZSByZXN1bHRzLgoKU2NhbGluZyBGdW5jdGlvbjoKKiBYaSAgICA9IEFuIGluc3RhbmNlIAoqIFhtZWFuID0gTWVhbi9BdmVyYWdlIG9mIGFsbCBzYW1wbGVzICAKKiBYc2QgICA9IFN0YW5kYXJkIERldmlhdGlvbiBvZiBvdXIgZGF0YSBzZXQgCiogWmkgICAgPSBTY2FsZWQgVmFsdWUgb2YgWGkKKiBaaSAgICA9IChYaSAtIFhtZWFuKSAvIFhzZCAKCmBgYHtyfQojIFN0YW5kYXJkIERldmlhdGlvbiBhbmQgTWVhbiBmb3IgU3RhbmRhcml6YXRpb24Kc2NhbGVfdGQgPC0gZnVuY3Rpb24oIFhpLCBYbWVhbiwgWHNkKSB7CiAgcmV0dXJuICggKFhpIC0gWG1lYW4pIC8gWHNkICkgIyBaaQp9CnRyYWluRGF0YV9zZF9TUyAgICAgICAgPC0gc2QodHJhaW5EYXRhU2V0WywiU1MiXSkKdHJhaW5EYXRhX3NkX1BvcyAgICAgICA8LSBzZCh0cmFpbkRhdGFTZXRbLCJQb3MxMHdydHNSTkFTdGFydCJdKQp0cmFpbkRhdGFfc2RfRGlzVGVybSAgIDwtIHNkKHRyYWluRGF0YVNldFssIkRpc3RUZXJtIl0pCnRyYWluRGF0YV9zZF9EaXMgICAgICAgPC0gc2QodHJhaW5EYXRhU2V0WywiRGlzdGFuY2UiXSkKdHJhaW5EYXRhX3NkX0Rvd25EaXMgICA8LSBzZCh0cmFpbkRhdGFTZXRbLCJEb3duRGlzdGFuY2UiXSkKCnRyYWluRGF0YV9tZWFuX1NTICAgICAgPC0gbWVhbih0cmFpbkRhdGFTZXRbLCJTUyJdKQp0cmFpbkRhdGFfbWVhbl9Qb3MgICAgIDwtIG1lYW4odHJhaW5EYXRhU2V0WywiUG9zMTB3cnRzUk5BU3RhcnQiXSkKdHJhaW5EYXRhX21lYW5fRGlzVGVybSA8LSBtZWFuKHRyYWluRGF0YVNldFssIkRpc3RUZXJtIl0pCnRyYWluRGF0YV9tZWFuX0RpcyAgICAgPC0gbWVhbih0cmFpbkRhdGFTZXRbLCJEaXN0YW5jZSJdKQp0cmFpbkRhdGFfbWVhbl9Eb3duRGlzIDwtIG1lYW4odHJhaW5EYXRhU2V0WywiRG93bkRpc3RhbmNlIl0pCgpzbHQyZGF0YV9zY2FsZWQgPC0gc2x0MmRhdGEKc2x0MmRhdGFfc2NhbGVkJFNTICAgICAgICAgICAgICAgIDwtIHNjYWxlX3RkKHNsdDJkYXRhX3NjYWxlZCRTUywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWVhbl9TUywgdHJhaW5EYXRhX3NkX1NTKQpzbHQyZGF0YV9zY2FsZWQkUG9zMTB3cnRzUk5BU3RhcnQgPC0gc2NhbGVfdGQoc2x0MmRhdGFfc2NhbGVkJFBvczEwd3J0c1JOQVN0YXJ0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21lYW5fUG9zLCB0cmFpbkRhdGFfc2RfUG9zKQpzbHQyZGF0YV9zY2FsZWQkRGlzdFRlcm0gICAgICAgICAgPC0gc2NhbGVfdGQoc2x0MmRhdGFfc2NhbGVkJERpc3RUZXJtLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9tZWFuX0Rpc1Rlcm0sIHRyYWluRGF0YV9zZF9EaXNUZXJtKQpzbHQyZGF0YV9zY2FsZWQkRGlzdGFuY2UgICAgICAgICAgPC0gc2NhbGVfdGQoc2x0MmRhdGFfc2NhbGVkJERpc3RhbmNlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21lYW5fRGlzLCB0cmFpbkRhdGFfc2RfRGlzKQpzbHQyZGF0YV9zY2FsZWQkRG93bkRpc3RhbmNlICAgICAgPC0gc2NhbGVfdGQoc2x0MmRhdGFfc2NhbGVkJERvd25EaXN0YW5jZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9tZWFuX0Rvd25EaXMsIHRyYWluRGF0YV9zZF9Eb3duRGlzKQpsdWRhdGFfc2NhbGVkIDwtIGx1ZGF0YQoKbHVkYXRhX3NjYWxlZCRTUyAgICAgICAgICAgICAgICA8LSBzY2FsZV90ZChsdWRhdGFfc2NhbGVkJFNTLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9tZWFuX1NTLCB0cmFpbkRhdGFfc2RfU1MpCmx1ZGF0YV9zY2FsZWQkUG9zMTB3cnRzUk5BU3RhcnQgPC0gc2NhbGVfdGQobHVkYXRhX3NjYWxlZCRQb3MxMHdydHNSTkFTdGFydCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9tZWFuX1BvcywgdHJhaW5EYXRhX3NkX1BvcykKbHVkYXRhX3NjYWxlZCREaXN0VGVybSAgICAgICAgICA8LSBzY2FsZV90ZChsdWRhdGFfc2NhbGVkJERpc3RUZXJtLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9tZWFuX0Rpc1Rlcm0sIHRyYWluRGF0YV9zZF9EaXNUZXJtKQpsdWRhdGFfc2NhbGVkJERpc3RhbmNlICAgICAgICAgIDwtIHNjYWxlX3RkKGx1ZGF0YV9zY2FsZWQkRGlzdGFuY2UsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWVhbl9EaXMsIHRyYWluRGF0YV9zZF9EaXMpCmx1ZGF0YV9zY2FsZWQkRG93bkRpc3RhbmNlICAgICAgPC0gc2NhbGVfdGQobHVkYXRhX3NjYWxlZCREb3duRGlzdGFuY2UsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWVhbl9Eb3duRGlzLCB0cmFpbkRhdGFfc2RfRG93bkRpcykKCnJiaW5kKCBoZWFkKHNsdDJkYXRhX3NjYWxlZCw1KSwgdGFpbChzbHQyZGF0YV9zY2FsZWQsNSkpCnJiaW5kKCBoZWFkKGx1ZGF0YV9zY2FsZWQsNSksIHRhaWwobHVkYXRhX3NjYWxlZCw1KSkKCmBgYAoKIyMjIDMuNyBOb3JtYWxpemUgdGVzdGluZyBEYXRhc2V0cwpJbiBhIHNpbWlsYXIgZmFzaGlvbiB0byB3aGF0IHdlIGRpZCBpbiAzLjYsIHdlIGFsc28gbmVlZCB0byB1c2UgdGhlIHRyYWluaW5nIGRhdGEncyBtYXggYW5kIG1pbiB2YWx1ZXMgdG8gcHJvcGVybHkgbm9ybWFsaXplIGVhY2ggZmVhdHVyZS4KCmBgYHtyfQpub3JtYWxpemVfdGQgPC0gZnVuY3Rpb24oeCwgbWluLCBtYXgpIHsKICByZXR1cm4gKCAoeCAtIG1pbikgLyAobWF4IC0gbWluKSApCn0KCnRyYWluRGF0YV9tYXhfU1MgICAgICA8LSBtYXgodHJhaW5EYXRhU2V0WywiU1MiXSkKdHJhaW5EYXRhX21heF9Qb3MgICAgIDwtIG1heCh0cmFpbkRhdGFTZXRbLCJQb3MxMHdydHNSTkFTdGFydCJdKQp0cmFpbkRhdGFfbWF4X0Rpc1Rlcm0gPC0gbWF4KHRyYWluRGF0YVNldFssIkRpc3RUZXJtIl0pCnRyYWluRGF0YV9tYXhfRGlzICAgICA8LSBtYXgodHJhaW5EYXRhU2V0WywiRGlzdGFuY2UiXSkKdHJhaW5EYXRhX21heF9Eb3duRGlzIDwtIG1heCh0cmFpbkRhdGFTZXRbLCJEb3duRGlzdGFuY2UiXSkKCnRyYWluRGF0YV9taW5fU1MgICAgICA8LSBtaW4odHJhaW5EYXRhU2V0WywiU1MiXSkKdHJhaW5EYXRhX21pbl9Qb3MgICAgIDwtIG1pbih0cmFpbkRhdGFTZXRbLCJQb3MxMHdydHNSTkFTdGFydCJdKQp0cmFpbkRhdGFfbWluX0Rpc1Rlcm0gPC0gbWluKHRyYWluRGF0YVNldFssIkRpc3RUZXJtIl0pCnRyYWluRGF0YV9taW5fRGlzICAgICA8LSBtaW4odHJhaW5EYXRhU2V0WywiRGlzdGFuY2UiXSkKdHJhaW5EYXRhX21pbl9Eb3duRGlzIDwtIG1pbih0cmFpbkRhdGFTZXRbLCJEb3duRGlzdGFuY2UiXSkKCnNsdDJkYXRhX25vcm0gPC0gc2x0MmRhdGEKc2x0MmRhdGFfbm9ybSRTUyAgICAgICAgICAgICAgICA8LSBub3JtYWxpemVfdGQoc2x0MmRhdGFfbm9ybSRTUywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWluX1NTLCB0cmFpbkRhdGFfbWF4X1NTKQpzbHQyZGF0YV9ub3JtJFBvczEwd3J0c1JOQVN0YXJ0IDwtIG5vcm1hbGl6ZV90ZChzbHQyZGF0YV9ub3JtJFBvczEwd3J0c1JOQVN0YXJ0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21pbl9Qb3MsIHRyYWluRGF0YV9tYXhfUG9zKQpzbHQyZGF0YV9ub3JtJERpc3RUZXJtICAgICAgICAgIDwtIG5vcm1hbGl6ZV90ZChzbHQyZGF0YV9ub3JtJERpc3RUZXJtLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21pbl9EaXNUZXJtLCB0cmFpbkRhdGFfbWF4X0Rpc1Rlcm0pCnNsdDJkYXRhX25vcm0kRGlzdGFuY2UgICAgICAgICAgPC0gbm9ybWFsaXplX3RkKHNsdDJkYXRhX25vcm0kRGlzdGFuY2UsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbkRhdGFfbWluX0RpcywgdHJhaW5EYXRhX21heF9EaXMpCnNsdDJkYXRhX25vcm0kRG93bkRpc3RhbmNlICAgICAgPC0gbm9ybWFsaXplX3RkKHNsdDJkYXRhX25vcm0kRG93bkRpc3RhbmNlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9taW5fRG93bkRpcywgdHJhaW5EYXRhX21heF9Eb3duRGlzKQpsdWRhdGFfbm9ybSA8LSBsdWRhdGEKbHVkYXRhX25vcm0kU1MgICAgICAgICAgICAgICAgPC0gbm9ybWFsaXplX3RkKGx1ZGF0YV9ub3JtJFNTLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9taW5fU1MsIHRyYWluRGF0YV9tYXhfU1MpCmx1ZGF0YV9ub3JtJFBvczEwd3J0c1JOQVN0YXJ0IDwtIG5vcm1hbGl6ZV90ZChsdWRhdGFfbm9ybSRQb3MxMHdydHNSTkFTdGFydCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9taW5fUG9zLCB0cmFpbkRhdGFfbWF4X1BvcykKbHVkYXRhX25vcm0kRGlzdFRlcm0gICAgICAgICAgPC0gbm9ybWFsaXplX3RkKGx1ZGF0YV9ub3JtJERpc3RUZXJtLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJhaW5EYXRhX21pbl9EaXNUZXJtLCB0cmFpbkRhdGFfbWF4X0Rpc1Rlcm0pCmx1ZGF0YV9ub3JtJERpc3RhbmNlICAgICAgICAgIDwtIG5vcm1hbGl6ZV90ZChsdWRhdGFfbm9ybSREaXN0YW5jZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9taW5fRGlzLCB0cmFpbkRhdGFfbWF4X0RpcykKbHVkYXRhX25vcm0kRG93bkRpc3RhbmNlICAgICAgPC0gbm9ybWFsaXplX3RkKGx1ZGF0YV9ub3JtJERvd25EaXN0YW5jZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRyYWluRGF0YV9taW5fRG93bkRpcywgdHJhaW5EYXRhX21heF9Eb3duRGlzKQoKcmJpbmQoIGhlYWQoc2x0MmRhdGFfbm9ybSw1KSwgdGFpbChzbHQyZGF0YV9ub3JtLDUpKQoKcmJpbmQoIGhlYWQobHVkYXRhX25vcm0sNSksIHRhaWwobHVkYXRhX25vcm0sNSkpCgpgYGAKCiMjIDQuIExvYWQgb3JpZ2luYWwgbW9kZWwKClRoZSBvcmlnaW5hbCBhcnRpY2xlLCB0aXRsZWQgIlByaW9yaXRpemluZyBib25hIGZpZGUgYmFjdGVyaWFsIHNtYWxsIFJOQXMgd2l0aCBtYWNoaW5lIGxlYXJuaW5nIGNsYXNzaWZpZXJzIiwgYW5kIHRoZSBzb3VyY2UgbWF0ZXJpYWxzKHRyYWluaW5nIGRhdGEsIHRlc3RpbmcgZGF0YSwgUiBzY3JpcHRzLCAucmRzIGZpbGUsIGV0Yy4pIGNhbiBhbGwgYmUgZm91bmQgaW4gW1BlZXJKXShodHRwczovL3BlZXJqLmNvbS9hcnRpY2xlcy82MzA0LykgdW5kZXIgdGhlIGZvbGxvd2luZyBbbGlua10oaHR0cHM6Ly9wZWVyai5jb20vYXJ0aWNsZXMvNjMwNC8pLgpUaGUgb3JpZ2luYWwgKi5yZHMqIGZpbGUgY2FuIGFsc28gYmUgZG93bmxvYWRlZCBmcm9tIFtnaXRodWJdKGh0dHBzOi8vZ2l0aHViLmNvbS9CaW9pbmZvcm1hdGljc0xhYkF0TVVOL3NSTkFSYW5raW5nKS4KCiMjIyA0LjEgTG9hZC9SZXRyYWluIE9yaWcgTW9kZWwKV2UgY2FuIGVpdGhlciByZXRyYWluIHRoZSBvcmlnaW5hbCBtb2RlbCwgb3IgbG9hZCB0aGUgLnJkcyBmaWxlIHByb3ZpZGVkIGluIHRoZSBvcmlnaW5hbCBhcnRpY2xlLiBGb3IgdGhlIHB1cnBvc2Ugb2Ygc2F2aW5nIHRpbWUsIHdlIHNpbXBseSBsb2FkZWQgdGhlIG1vZGVsLgoKYGBge3J9Cm9yaWdSRiA8LSByZWFkUkRTKCJSRl9jbGFzc2lmaWVyNHNSTkEucmRzIikKYGBgCgojIyMgNC4yIFJldHJhaW4gbmV3IG1vZGVscyB3aXRoIFNjYWxlZCBhbmQgTm9ybWFsaXplZCBEYXRhIApTaW5jZSBzb21lIG9mIHRoZSBJTSBtb2RlbHMgcmVxdWlyZWQgc2NhbGluZyBvciBub3JtYWxpemVkIGRhdGEgdG8gcHJvcGVybHkgZnVuY3Rpb24sIHdlIG5lZWRlZCB0byBjcmVhdGUgMiBuZXcgbW9kZWxzIGluIHRoZSBzYW1lIHdheSBhcyB0aGUgb25lIGxvYWRlZCBpbiA0LjEuIAoKYGBge3J9CnR1bmVSRihkYXRhU2V0VHJhaW5YWyxjKDE6NyldLCB5ID0gZmFjdG9yKGRhdGFTZXRUcmFpbllbLDJdKSwgbnRyZWVUcnkgPSA0MDAsIG10cnlTdGFydCA9IDIpCnNldC5zZWVkKDEyMzQpCm9yaWdSRl9zY2FsZWQgPC0gcmFuZG9tRm9yZXN0KHggPSBkYXRhU2V0VHJhaW5fc2NhbGVkWyxjKDE6NyldLCB5ID0gZmFjdG9yKGRhdGFTZXRUcmFpbllbLDJdKSwgbXRyeSA9IDIsIG50cmVlID0gNDAwLCBpbXBvcnRhbmNlID0gVFJVRSkKb3JpZ1JGX25vcm0gPC0gcmFuZG9tRm9yZXN0KHggPSBkYXRhU2V0VHJhaW5fbm9ybVssYygxOjcpXSwgeSA9IGZhY3RvcihkYXRhU2V0VHJhaW5ZWywyXSksIG10cnkgPSAyLCBudHJlZSA9IDQwMCwgaW1wb3J0YW5jZSA9IFRSVUUpCmBgYAoKIyBJSSkgVFJBSU4gVEhFIE5FVyBIMk8gTU9ERUwgClNpbmNlIHdlIGFyZSBnb2luZyB0byB1c2UgdGhlIGgybyBsaWJyYXJ5IHRvIHBlcmZvcm0gc29tZSBvZiBvdXIgbWFjaGluZSBsZWFybmluZyBpbnRlcnByZXRhYmlsaXR5LCB3ZSBuZWVkIHRvIHRyYWluIGFuIGVxdWl2YWxlbnQgb25lLCB1c2luZyB0aGUgaDJvIHRvb2xzLgoKIyMgNS4gSW50aWFsaXplIEgyTyBhbmQgbG9hZCB0cmFpbmluZyBkYXRhIAoKYGBge3J9Cmgyby5pbml0KCAgICAgICAgICAgICAgIyBJbml0aWFsaXplIGgybwogIG50aHJlYWRzID0gLTEsICAgICAgICMgLTEgPSB1c2UgYWxsIGF2YWlsYWJsZSB0aHJlYWRzCiAgbWF4X21lbV9zaXplID0gIjhHIiAgIyBzcGVjaWZ5IHRoZSBhbW91bnQgb2YgbWVtb3J5IHRvIHVzZQopCmgyby5yZW1vdmVBbGwoKSAgICAgICAgIyBjbGVhbiB1cCB0aGUgc3lzdGVtLCBoMm8td2lzZSAoaWUuIGtpbGwgYW55IHJ1bm5pbmcgaDJvIGNsdXN0ZXJzKQpoMm8ubm9fcHJvZ3Jlc3MoKSAKdHJhaW5EYXRhIDwtIGFzLmgybyh0cmFpbkRhdGFTZXQpIAp0cmFpbkRhdGFfc2NhbGVkIDwtIGFzLmgybyhkYXRhU2V0VHJhaW5fc2NhbGVkKSAKdHJhaW5EYXRhX25vcm0gPC0gYXMuaDJvKGRhdGFTZXRUcmFpbl9ub3JtKSAKYGBgCgojIyA2LiBCdWlsZCBtb2RlbAojIyMgNi4xIEJ1aWxkIG1vZGVsIHdpdGggT3JpZyBUcmFpbmluZyBkYXRhCmBgYHtyfQpyZmgybyA8LSBoMm8ucmFuZG9tRm9yZXN0KCAKICAgICAgICAgICAgIHRyYWluaW5nX2ZyYW1lID0gdHJhaW5EYXRhLCAjIHRyYWluaW5nIHVzaW5nIHRoZSBub3JtYWwgZGF0YQogICAgICAgICAgICAgeCA9IDE6NywgICAgICAgICAgICAgICAgICAgICMgZmVhdHVyZXMgdG8gdXNlIHRvIGdlbmVyYXRlIHRoZSBwcmVkaWN0aW9uCiAgICAgICAgICAgICB5ID0gOSwgICAgICAgICAgICAgICAgICAgICAgIyBDbGFzcyB0eXBlIC0+IHdoYXQgd2Ugd2FudCB0byBwcmVkaWN0CiAgICAgICAgICAgICBtb2RlbF9pZCA9ICJyZjFfc1JOQSIsICAgICAgIyBuYW1lIG9mIG1vZGVsIGluIGgybwogICAgICAgICAgICAgbnRyZWVzID0gNDAwLCAgICAgICAgICAgICAgICMgbWF4IG51bWJlciBvZiB0cmVlcyAgCiAgICAgICAgICAgICBzZWVkID0gMTIzNCwgICAgICAgICAgICAgICAgIyBzZWVkLCBoYXMgdG8gYmUgc2V0IFdJVEhJTiB0aGUgaDJvIGZ1bmN0aW9uIGFuZCBpdCdzIHN1cHBvc2VkIHRvIGJlIGRpZmZlcmVudCBmcm9tICJSJ3Mgc2VlZCIsIHNvIHJlc3VsdHMgbWlnaHQgbm90IGJlIGV4YWN0bHkgdGhlIHNhbWUgYXMgb3JpZyBtb2RlbCwgYnV0IHNob3VsZCBiZSBzaW1pbGFyIGVub3VnaAogICAgICAgICAgICAgbXRyaWVzID0gMiwgICAgICAgICAgICAgICAgICMgU2FtZSBhcyBvcmlnaW5hbCBtb2RlbCAKICAgICAgICAgICAgIG1heF9kZXB0aCA9IDMwCikKYGBgCgojIyMgNi4yIEJ1aWxkIG1vZGVsIHdpdGggU2NhbGVkIFRyYWluaW5nIGRhdGEgCmBgYHtyfQpyZmgyb19zY2FsZWQgPC0gaDJvLnJhbmRvbUZvcmVzdCggCiAgdHJhaW5pbmdfZnJhbWUgPSB0cmFpbkRhdGFfc2NhbGVkLAogIHggPSAxOjcsICAgICAgICAgICAgICAgIAogIHkgPSA5LCAgICAgICAgICAgICAgICAgICAgIAogIG1vZGVsX2lkID0gInJmMl9zUk5BIiwgICAgIAogIG50cmVlcyA9IDQwMCwgICAgICAgICAgICAgIAogIHNlZWQgPSAxMjM0LCAgICAgICAgICAgICAgCiAgbXRyaWVzID0gMiwgICAgICAgICAgICAgICAgCiAgbWF4X2RlcHRoID0gMzAKKQpgYGAKCiMjIyA2LjMgQnVpbGQgbW9kZWwgd2l0aCBOb3JtIFRyYWluaW5nIGRhdGEKYGBge3J9CnJmaDJvX25vcm0gPC0gaDJvLnJhbmRvbUZvcmVzdCggCiAgdHJhaW5pbmdfZnJhbWUgPSB0cmFpbkRhdGFfbm9ybSwgCiAgeCA9IDE6NywgICAgICAgICAgICAgICAgICAKICB5ID0gOSwgICAgICAgICAgICAgICAgICAgIAogIG1vZGVsX2lkID0gInJmM19zUk5BIiwgICAgIAogIG50cmVlcyA9IDQwMCwgICAgICAgICAgICAgICAKICBzZWVkID0gMTIzNCwgICAgICAgICAgICAgICAKICBtdHJpZXMgPSAyLCAgICAgICAgICAgICAgICAgIAogIG1heF9kZXB0aCA9IDMwCikKYGBgCgojIyMgNi40IEJPTlVTOiBCdWlsZCBHTE0gbW9kZWwKQXMgd2Ugd2VyZSBkb2luZyB0ZXN0cyB3aXRoIExJTUUsIHdlIGRpc2NvdmVyZWQgdGhhdCBpdCB5aWVsZGVkIGlycmVndWxhciByZXNwb25zZXMsIGFuZCBhcyBhIHJlc3VsdCwgdGhlIGV4cGxhbmF0aW9ucyBjb3VsZCBub3QgYmUgdHJ1c3RlZCBhdCB0aGUgdGltZS4gSXQgc2VlbXMgdG8gYmUgYSBrbm93biBpc3N1ZSB0aGF0IExJTUUgaXMgbm90IHBlcmZlY3QgZm9yIGV2ZXJ5IG1vZGVsLCBhbmQgdGhhdCBpdCBzdHJ1Z2dsZXMgd2l0aCBkYXRhIHRoYXQgaXMgaGlnaGx5IGRpdmVyc2UuIEluIHRoaXMgdXNlIGNhc2UsIG91ciBkYXRhIGNvbnRhaW5zIG1peGVkIG51bWVyaWNhbCBhbmQgY2F0ZWdvcmljYWwgZmVhdHVyZXMsIHNpZ25pZmljYW50bHkgZGlmZmVyZW50IHNjYWxlcyBhbmQgbWFnbml0dWRlcyBpbiBvdXIgbnVtZXJjaWFsIGZlYXR1cmVzLCBhbmQgYW4gaW5iYWxhbmNlZCBkYXRhIHNldHMgdGhhdCBmYXZvcnMgcHJlZGljdGlvbnMgb2Ygc1JOQXMgYmVpbmcgZmFsc2UuIFNpbmNlIGxpbWUgY3JlYXRlcyBhIGxvY2FsIGludGVycHJldGFibGUgbGluZWFyIG1vZGVsIGZvciBpdHMgZXhwbGFuYXRpb25zLCB0aGUgR0xNIGNvbnN0cnVjdGVkIGhlcmUgd2FzIHNpbXBseSB0byB2ZXJpZnkgdGhlIGh5cG90aGVzaXMgdGhhdCBhIGxpbmVhciBtb2RlbCB3b3VsZCBiZSB1bmFibGUgdG8gZXhwbGFpbiB0aGUgZGF0YS4gVGhlICJ1bmJhbGFuY2VkLW5lc3MiIHNlZW1zIHRvIGNyZWF0ZSBtb2RlbHMgd2hlcmUgdGhlIGZlYXR1cmVzIHRlbmQgYWx3YXlzIHN1cHBvcnQgYSByZXN1bHQgb2YgImZhbHNlIiwgd2hpbGUgY2xhaW1pbmcgdGhhdCBhbGwgdGhlIGZlYXR1cmVzIGFyZSBhZ2FpbnN0IGEgcmVzdWx0IG9mIHRydWUsIHdoaWNoIHdpbGwgYmUgZXZpY2VuY2VkIGluIHNlY3Rpb24gMTUuMS4KCmBgYHtyfQpnbG1oMm8gPC0gaDJvLmdsbSggIyBodHRwczovL2RvY3MuaDJvLmFpL2gyby9sYXRlc3Qtc3RhYmxlL2gyby1kb2NzL2RhdGEtc2NpZW5jZS9nbG0uaHRtbCAKICB0cmFpbmluZ19mcmFtZSA9IHRyYWluRGF0YSwgIyB0cmFpbmluZyB1c2luZyB0aGUgbm9ybWFsIGRhdGEKICB4ID0gMTo3LCAgICAgICAgICAgICAgICAgICAjIGZlYXR1cmVzIHRvIHVzZSB0byBnZW5lcmF0ZSB0aGUgcHJlZGljdGlvbgogIHkgPSA5LCAgICAgICAgICAgICAgICAgICAgICMgQ2xhc3MgdHlwZSAtPiB3aGF0IHdlIHdhbnQgdG8gcHJlZGljdAogIG1vZGVsX2lkID0gImdsbV9zUk5BIiwgICAgICMgbmFtZSBvZiBtb2RlbCBpbiBoMm8KICBzZWVkID0gMTIzNCwKICBmYW1pbHkgPSAiYmlub21pYWwiCikKCmdsbWgyb19wZXJmIDwtIGgyby5wZXJmb3JtYW5jZShnbG1oMm8pCmdsbWgyb19wZXJmCmgyby5jb2VmKGdsbWgybykKaDJvLmNvZWZfbm9ybShnbG1oMm8pCmdsbWgyb0Btb2RlbCRjb2VmZmljaWVudHNfdGFibGUKCmgyby5yMihnbG1oMm8pICMgUl4yIHZhbHVlIGlzIHJlYWxseSBsb3csIGFuZCBoYXZpbmcgYSBNYXggUmVjYWxsIGJlaW5nIHJlYWxseSBsb3csIG1lYW5pbmcgdGhhdCBMSU1FIGlzIGFsc28gcHJlZGljdGluZyB0aGF0IGV2ZXJ5dGhpbmcgc2hvdWxkIGJlIGZhbHNlCmBgYAoKIyMgNy4gUHJldmlldyBIMk8ncyBSRiAKVGhpcyBpcyB0aGUgcGVyZm9ybWFuY2Ugd2l0aCB0aGUgT09CIGVycm9yLCBiYXNlZCBvbiB0aGUgdHJhaW5pbmcgcHJvY2VzcyAoaWUgdHJhaW5pbmcgZGF0YSkKCmBgYHtyfQpyZmgybwpyZmgyb0Btb2RlbCR2YXJpYWJsZV9pbXBvcnRhbmNlcwpoMm8udmFyaW1wX3Bsb3QocmZoMm8pCnJmaDJvX3RyYWluaW5nX3BlZm9ybWFuY2UgPC0gaDJvLnBlcmZvcm1hbmNlKHJmaDJvKQpyZmgyb190cmFpbmluZ19wZWZvcm1hbmNlCmBgYAoKCiMgSUlJKSBDT01QQVJFIE1PREVMJ1MgUFJFRElDVElPTlMKSW4gdGhpcyBzZWN0aW9uIHdlJ2xsIGdldCB0aGUgcHJlZGljdGlvbnMgYWxsIHRoZSBtb2RlbHMgZ2VuZXJhdGUgb24gdGhlIHRlc3RpbmcgZGF0YSBMVSBhbmQgU0xUMi4gVGhlIGdvYWwgb2YgdGhpcyBzZWN0aW9uIGlzIG5vdCB0byBwcm92ZSB3aGljaCBtb2RlbCBpcyB0aGUgYmVzdCwgYnV0IHRvIHByb3ZlIHRoYXQgdGhlIG1vZGVscyBhcmUgc2ltaWxhciBlbm91Z2ggdG8gdGhlIHBvaW50IHRoYXQgdGhlIGFuYWx5c2lzIG9mIHRoZSBIMk8gbW9kZWwgd2lsbCBiZSBhIHZhbGlkIGFuYWxvZ3kgbW9kZWwgZm9yIHRoZSBvcmlnaW5hbCBSRiBtb2RlbC4gCgojIyA4LiBHZXQgVGVzdGluZyBEYXRhIFByZWRpY3Rpb25zIGZyb20gdGhlIE1vZGVscwpgYGB7cn0Kb3JpZ1JGX2x1X3ByZWQgPC0gcHJlZGljdChvcmlnUkYsIGx1ZGF0YVssLThdLCB0eXBlID0gInByb2IiKQpvcmlnUkZfc2x0Ml9wcmVkIDwtIHByZWRpY3Qob3JpZ1JGLCBzbHQyZGF0YVssLThdLCB0eXBlID0gInByb2IiKQoKb3JpZ1JGX3NjYWxlZF9sdV9wcmVkIDwtIHByZWRpY3Qob3JpZ1JGX3NjYWxlZCwgbHVkYXRhX3NjYWxlZFssLThdLCB0eXBlID0gInByb2IiKQpvcmlnUkZfc2NhbGVkX3NsdDJfcHJlZCA8LSBwcmVkaWN0KG9yaWdSRl9zY2FsZWQsIHNsdDJkYXRhX3NjYWxlZFssLThdLCB0eXBlID0gInByb2IiKQoKb3JpZ1JGX25vcm1fbHVfcHJlZCA8LSBwcmVkaWN0KG9yaWdSRl9ub3JtLCBsdWRhdGFfbm9ybVssLThdLCB0eXBlID0gInByb2IiKQpvcmlnUkZfbm9ybV9zbHQyX3ByZWQgPC0gcHJlZGljdChvcmlnUkZfbm9ybSwgc2x0MmRhdGFfbm9ybVssLThdLCB0eXBlID0gInByb2IiKQoKcmZoMm9fc2x0Ml9wcmVkIDwtIGgyby5wcmVkaWN0KG9iamVjdCA9IHJmaDJvLCBuZXdkYXRhID0gYXMuaDJvKHNsdDJkYXRhWywtOF0pICkKcmZoMm9fbHVfcHJlZCA8LSBoMm8ucHJlZGljdChvYmplY3QgPSByZmgybywgbmV3ZGF0YSA9IGFzLmgybyhsdWRhdGFbLC04XSkgKQoKcmZoMm9fc2NhbGVkX3NsdDJfcHJlZCA8LSBoMm8ucHJlZGljdChvYmplY3QgPSByZmgyb19zY2FsZWQsIG5ld2RhdGEgPSBhcy5oMm8oc2x0MmRhdGFfc2NhbGVkWywtOF0pICkKcmZoMm9fc2NhbGVkX2x1X3ByZWQgPC0gaDJvLnByZWRpY3Qob2JqZWN0ID0gcmZoMm9fc2NhbGVkLCBuZXdkYXRhID0gYXMuaDJvKGx1ZGF0YV9zY2FsZWRbLC04XSkgKQoKcmZoMm9fbm9ybV9zbHQyX3ByZWQgPC0gaDJvLnByZWRpY3Qob2JqZWN0ID0gcmZoMm9fbm9ybSwgbmV3ZGF0YSA9IGFzLmgybyhzbHQyZGF0YV9ub3JtWywtOF0pICkKcmZoMm9fbm9ybV9sdV9wcmVkIDwtIGgyby5wcmVkaWN0KG9iamVjdCA9IHJmaDJvX25vcm0sIG5ld2RhdGEgPSBhcy5oMm8obHVkYXRhX25vcm1bLC04XSkgKQoKZ2xtX3NsdDJfcHJlZCA8LSBoMm8ucHJlZGljdChvYmplY3QgPSBnbG1oMm8sIG5ld2RhdGEgPSBhcy5oMm8oc2x0MmRhdGFbLC04XSkgKQpnbG1fbHVfcHJlZCA8LSBoMm8ucHJlZGljdChvYmplY3QgPSBnbG1oMm8sIG5ld2RhdGEgPSBhcy5oMm8obHVkYXRhWywtOF0pICkKYGBgCgojIyA5LiBDcmVhdGUgYSBjb21wYXJpc29uIGZ1bmN0aW9uIApUaGlzIGZ1bmN0aW9uIGlzIHRvIHNpbXBsaWZ5IHRoZSBjb21wYXJpc29uIGJldHdlZW4gdGhlIHByZWRpY3Rpb25zIGZyb20gdGhlIE9yaWdpbmFsIFJGIG1vZGVsIGFuZCB0aGUgSDJPIFJGIG1vZGVsLiBMYXRlciBvbiB0aGlzLCB0aGlzIGZ1bmN0aW9uIHdpbGwgYmUgdXNlZCB0byBjb21wYXJlIHRoZSBwcmVkaWN0aW9ucyBiZXR3ZWVuIHRoZSBPcmlnUkYgbW9kZWwgYW5kIHRoZSBvdGhlciBtb2RlbHMuIAoKYGBge3J9CiMgVGhpcyBmdW5jdGlvbiB3aWxsIGJlIHVzZWQgbGF0ZXIgb24sIGFuZCAKIyAgIGEgPSB0aGUgb3JpZ2luYWwgbW9kZWwncyBwcmVkaWN0aW9ucwojICAgYiA9IHRoZSBhbHRlcm5hdGUgbW9kZWwncyBwcmVkaWN0aW9ucwojICAgYyA9IHRoZSBjb3JyZWN0IHByZWRpY3Rpb24KCmNvbXBhcmVBbnN3ZXJzIDwtIGZ1bmN0aW9uKGEsYixjKXsKICBpZiAoIGEgPT0gYyAmIGIgPT0gYyApIAogICAgICB7IHJlcz0iQk9USF9SSUdIVCIgfQogIGVsc2UgaWYgKCBhID09IGMgJiBiICE9IGMgKSAKICAgICAgeyByZXM9Ik9ubHlBIiAgfQogIGVsc2UgaWYgKCBhICE9IGMgJiBiID09IGMgKSAKICAgICAgeyByZXM9Ik9ubHlCIiAgfQogIGVsc2UgCiAgICAgIHsgcmVzPSJCT1RIX1dST05HIiAgfQogIHJldHVybihyZXMpCn0KYGBgCgojIyAxMC4gU0xUMiBjb21wYXJpc29ucyAKIyMjIDEwLjEgQnVpbGQgU0xUMiBQcmVkaWN0aW9ucyB0YWJsZSAKCmBgYHtyfQpzbHQyX3ByZWRpY3Rpb25zIDwtICBjYmluZChhcy5kYXRhLmZyYW1lKHNsdDJkYXRhKSwgICAgICAgICAgICAgICAjIElucHV0CiAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzLmRhdGEuZnJhbWUob3JpZ1JGX3NsdDJfcHJlZFssMl0pLCAgICMgT3JpZyBSRiBwcmVkaWN0aW9ucyAKICAgICAgICAgICAgICAgICAgICAgICAgICAgYXMuZGF0YS5mcmFtZShvcmlnUkZfc2NhbGVkX3NsdDJfcHJlZFssMl0pLCAgICMgT3JpZyBSRiBTY2FsZWQgcHJlZGljdGlvbnMgCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzLmRhdGEuZnJhbWUob3JpZ1JGX25vcm1fc2x0Ml9wcmVkWywyXSksICAgIyBPcmlnIFJGIE5vcm1hbGl6ZWQgcHJlZGljdGlvbnMgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgYXMuZGF0YS5mcmFtZShyZmgyb19zbHQyX3ByZWRbLDNdKSwgICAgICAgICMgSDJPIFJGIHByZWRpY3Rpb25zCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzLmRhdGEuZnJhbWUocmZoMm9fc2NhbGVkX3NsdDJfcHJlZFssM10pLCAjIFNjYWxlZCBSRiBwcmVkaWN0aW9ucwogICAgICAgICAgICAgICAgICAgICAgICAgICBhcy5kYXRhLmZyYW1lKHJmaDJvX25vcm1fc2x0Ml9wcmVkWywzXSkgICAgIyBOb3JtYWxpemVkIFJGIHByZWRpY3Rpb25zCiAgICAgICAgICAgICAgICAgICAgICAgICAgICkKCmNvbG5hbWVzKHNsdDJfcHJlZGljdGlvbnMpIDwtIGMoIlNTIiwgIlBvczEwd3J0c1JOQVN0YXJ0IiwgIkRpc3RUZXJtIiwgIkRpc3RhbmNlIiwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgInNhbWVTdHJhbmQiLCAiRG93bkRpc3RhbmNlIiwgInNhbWVEb3duU3RyYW5kIiwgIkNsYXNzIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiT3JpZ1JGX1AiLCJPcmlnUkZfc2NhbGVkX1AiLCJPcmlnUkZfbm9ybV9QIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiUkZfSDJPX1AiLCJSRkgyT19zY2FsZWRfUCIsIlJGSDJPX25vcm1fUCIpCgpzbHQyX3ByZWRpY3Rpb25zJG9yaWdQcmVkcyA8LSBpZmVsc2UoIHNsdDJfcHJlZGljdGlvbnMkT3JpZ1JGX1AgPj0gMC41LCAxLCAwKSAKc2x0Ml9wcmVkaWN0aW9ucyRvcmlnU2NhbGVkUHJlZHMgPC0gaWZlbHNlKCBzbHQyX3ByZWRpY3Rpb25zJE9yaWdSRl9zY2FsZWRfUCA+PSAwLjUsIDEsIDApIApzbHQyX3ByZWRpY3Rpb25zJG9yaWdOb3JtUHJlZHMgPC0gaWZlbHNlKCBzbHQyX3ByZWRpY3Rpb25zJE9yaWdSRl9ub3JtX1AgPj0gMC41LCAxLCAwKSAKc2x0Ml9wcmVkaWN0aW9ucyRoMm9QcmVkcyAgPC0gaWZlbHNlKCBzbHQyX3ByZWRpY3Rpb25zJFJGX0gyT19QID49IDAuNSwgMSwgMCkgCnNsdDJfcHJlZGljdGlvbnMkaDJvU2NhbGVkUHJlZHMgIDwtIGlmZWxzZSggc2x0Ml9wcmVkaWN0aW9ucyRSRkgyT19zY2FsZWRfUCA+PSAwLjUsIDEsIDApIApzbHQyX3ByZWRpY3Rpb25zJGgyb05vcm1QcmVkcyAgPC0gaWZlbHNlKCBzbHQyX3ByZWRpY3Rpb25zJFJGSDJPX25vcm1fUCA+PSAwLjUsIDEsIDApICAKCnNsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPIDwtIE5BCnNsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA8LSBOQQpzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtIDwtIE5BCnNsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkIDwtIE5BCnNsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPTm9ybSA8LSBOQQoKZm9yKCBpIGluIDE6bnJvdyhzbHQyX3ByZWRpY3Rpb25zKSApewogICMgQmFzZWQgb24gdGhlIHdheSB3ZSBhcmUgZmVlZGluZyB0aGUgdmFsdWVzIHRvIHRoZSBjb21wYXJlQW5zd2VycyBmdW5jdGlvbiwgCiAgIyBTaW1pbGlhcml0aWVzID0gT25seUEgbWVhbnMgdGhhdCBvbmx5IHRoZSBPcmlnIFJGIGdvdCB0aGUgcmlnaHQgYW5zd2VyCiAgIyBTaW1pbGlhcml0aWVzID0gT25seUIgbWVhbnMgdGhhdCBvbmx5IHRoZSBvdGhlciBSRiBnb3QgdGhlIHJpZ2h0IGFuc3dlcgogICMgQWxsIG90aGVyIGFuc3dlcnMgd2lsbCB0ZWxsIHVzIHdoZXJlIGJvdGggbW9kZWxzIHdlcmUgcmlnaHQoIkJPVEhfUklHSFQiKSwgb3IKICAjIHdyb25nICgiQk9USF9XUk9ORyIpCiAgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ZzT3JpZ1NjYWxlZCA8LSBjb21wYXJlQW5zd2VycygKICAgIHNsdDJfcHJlZGljdGlvbnNbaSxdJG9yaWdQcmVkcywgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kb3JpZ1NjYWxlZFByZWRzLHNsdDJfcHJlZGljdGlvbnNbaSxdJENsYXNzICkKICBzbHQyX3ByZWRpY3Rpb25zW2ksXSRvcmlnVnNPcmlnTm9ybSA8LSBjb21wYXJlQW5zd2VycygKICAgIHNsdDJfcHJlZGljdGlvbnNbaSxdJG9yaWdQcmVkcywgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kb3JpZ05vcm1QcmVkcywgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kQ2xhc3MgKQogIHNsdDJfcHJlZGljdGlvbnNbaSxdJG9yaWdWc0gyTyA8LSBjb21wYXJlQW5zd2VycygKICAgIHNsdDJfcHJlZGljdGlvbnNbaSxdJG9yaWdQcmVkcywgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kaDJvUHJlZHMsIHNsdDJfcHJlZGljdGlvbnNbaSxdJENsYXNzICkKICBzbHQyX3ByZWRpY3Rpb25zW2ksXSRvcmlnVnNIMk9TY2FsZWQgPC0gY29tcGFyZUFuc3dlcnMoCiAgICBzbHQyX3ByZWRpY3Rpb25zW2ksXSRvcmlnUHJlZHMsIHNsdDJfcHJlZGljdGlvbnNbaSxdJGgyb1NjYWxlZFByZWRzLCBzbHQyX3ByZWRpY3Rpb25zW2ksXSRDbGFzcyApCiAgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ZzSDJPTm9ybSA8LSBjb21wYXJlQW5zd2VycygKICAgIHNsdDJfcHJlZGljdGlvbnNbaSxdJG9yaWdQcmVkcywgc2x0Ml9wcmVkaWN0aW9uc1tpLF0kaDJvTm9ybVByZWRzLCBzbHQyX3ByZWRpY3Rpb25zW2ksXSRDbGFzcyApCn0gCgpyYmluZCggaGVhZChzbHQyX3ByZWRpY3Rpb25zLCA1KSwgdGFpbChzbHQyX3ByZWRpY3Rpb25zLCA1KSApCgpzdHIoc2x0Ml9wcmVkaWN0aW9ucykKCmBgYAoKQXMgYW4gYWRkaXRpb25hbCBjaGVjaywgd2UgYWxzbyByZXZpZXdlZCB0aGUgY29ycmVsYXRpb25zIGJldHdlZW4gdGhlIE9yaWdSRiBwcmVkaWN0aW9ucywgYW5kIHRoZSBhbHRlcm5hdGl2ZSBtb2RlbHMncyBwcmVkaWN0aW9ucyB3aXRoIHRoZSBTTFQyIHByZWRpY3Rpb25zLiAKCmBgYHtyfQpjb3Ioc2x0Ml9wcmVkaWN0aW9uc1ssIk9yaWdSRl9QIl0sc2x0Ml9wcmVkaWN0aW9uc1ssIlJGX0gyT19QIl0pIApjb3Ioc2x0Ml9wcmVkaWN0aW9uc1ssIk9yaWdSRl9QIl0sc2x0Ml9wcmVkaWN0aW9uc1ssIk9yaWdSRl9zY2FsZWRfUCJdKQpjb3Ioc2x0Ml9wcmVkaWN0aW9uc1ssIk9yaWdSRl9QIl0sc2x0Ml9wcmVkaWN0aW9uc1ssIk9yaWdSRl9ub3JtX1AiXSkKY29yKHNsdDJfcHJlZGljdGlvbnNbLCJPcmlnUkZfUCJdLHNsdDJfcHJlZGljdGlvbnNbLCJSRkgyT19zY2FsZWRfUCJdKSAKY29yKHNsdDJfcHJlZGljdGlvbnNbLCJPcmlnUkZfUCJdLHNsdDJfcHJlZGljdGlvbnNbLCJSRkgyT19ub3JtX1AiXSkKYGBgCgojIyMgMTAuMiBDb21wYXJlIE9yaWdSRiB2cyB0aGUgb3RoZXIgTW9kZWxzIHdpdGggdGhlIFNMVDIgZGF0YQpIZXJlLCB3ZSBhcmUgc2ltcGx5IHRha2luZyBhIGxvb2sgYXQgaG93IG1hbnkgaW5zdGFuY2VzIHRoZSBPcmlnUkYgbW9kZWwgZ290IGNvcnJlY3QgdnMgdGhlIG90aGVyIG1vZGVscywgYW5kIGhvdyBtYW55IGluc3RhbmNlcyBib3RoIG1vZGVscyBwcmVkaWN0ZWQgY29ycmVjdGx5IG9yIHdyb25nLgogICogQSB2YWx1ZSBvZiAqT25seUEqIG1lYW5zIHRoYXQgb25seSB0aGUgb3JpZyBSRiBnb3QgdGhlIHJpZ2h0IGFuc3dlcgogICogQSB2YWx1ZSBvZiAqT25seUIqIG1lYW5zIHRoYXQgb25seSB0aGUgb3RoZXIgUkYgZ290IHRoZSByaWdodCBhbnN3ZXIKICAqIEFsbCBvdGhlciBhbnN3ZXJzIHdpbGwgdGVsbCB1cyB3aGVyZSBib3RoIG1vZGVscyB3ZXJlIHJpZ2h0ICooIkJPVEhfUklHSFQiKSogb3Igd3JvbmcgKigiQk9USF9XUk9ORyIpKgoKIyMjIyAxMC4yLjIgQ29tcGFyZSBPcmlnUkYgdnMgT3JpZ1JGIFNjYWxlZCBNb2RlbCAKYGBge3J9Cm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJPbmx5QSIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJPbmx5QiIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJCT1RIX1dST05HIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdTY2FsZWQgPT0gIkJPVEhfUklHSFQiLF0gKQpgYGAKCiMjIyMgMTAuMi4zIENvbXBhcmUgT3JpZ1JGIHZzIE9yaWdSRiBOb3JtIE1vZGVsIApgYGB7cn0KbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtID09ICJPbmx5QSIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnTm9ybSA9PSAiT25seUIiLF0gKQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ05vcm0gPT0gIkJPVEhfV1JPTkciLF0gKQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ05vcm0gPT0gIkJPVEhfUklHSFQiLF0gKQpgYGAKCiMjIyMgMTAuMi40IENvbXBhcmUgT3JpZ1JGIHZzIEgyTyBNb2RlbCAKYGBge3J9Cm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk8gPT0gIk9ubHlBIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiT25seUIiLF0gKQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPID09ICJCT1RIX1dST05HIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiQk9USF9SSUdIVCIsXSApCmBgYAoKIyMjIyAxMC4yLjUgQ29tcGFyZSBPcmlnUkYgdnMgSDJPIE1vZGVsIFNjYWxlZCAKCmBgYHtyfQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkID09ICJPbmx5QSIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk9TY2FsZWQgPT0gIk9ubHlCIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT1NjYWxlZCA9PSAiQk9USF9XUk9ORyIsXSApCm5yb3coIHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk9TY2FsZWQgPT0gIkJPVEhfUklHSFQiLF0gKQpgYGAKCiMjIyMgMTAuMi42IENvbXBhcmUgT3JpZ1JGIHZzIEgyTyBNb2RlbCBOb3JtYWxpemVkIApgYGB7cn0KbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIk9ubHlBIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIk9ubHlCIixdICkKbnJvdyggc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIkJPVEhfV1JPTkciLF0gKQpucm93KCBzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPTm9ybSA9PSAiQk9USF9SSUdIVCIsXSApCmBgYAoKIyMgMTEuIExVIENvbXBhcmlzb25zIAojIyMgMTEuMSBCdWlsZCBMVSBQcmVkaWN0aW9ucyB0YWJsZSAKCmBgYHtyfQpsdV9wcmVkaWN0aW9ucyA8LSAgY2JpbmQoYXMuZGF0YS5mcmFtZShsdWRhdGEpLCAgICAgICAgICAgICAgICMgSW5wdXQKICAgICAgICAgICAgICAgICAgICAgICAgICAgYXMuZGF0YS5mcmFtZShvcmlnUkZfbHVfcHJlZFssMl0pLCAgICMgT3JpZyBSRiBwcmVkaWN0aW9ucyAKICAgICAgICAgICAgICAgICAgICAgICAgICAgYXMuZGF0YS5mcmFtZShvcmlnUkZfc2NhbGVkX2x1X3ByZWRbLDJdKSwgICAjIE9yaWcgUkYgU2NhbGVkIHByZWRpY3Rpb25zIAogICAgICAgICAgICAgICAgICAgICAgICAgICBhcy5kYXRhLmZyYW1lKG9yaWdSRl9ub3JtX2x1X3ByZWRbLDJdKSwgICAjIE9yaWcgUkYgTm9ybWFsaXplZCBwcmVkaWN0aW9ucyAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICBhcy5kYXRhLmZyYW1lKHJmaDJvX2x1X3ByZWRbLDNdKSwgICAgICAgICMgSDJPIFJGIHByZWRpY3Rpb25zCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGFzLmRhdGEuZnJhbWUocmZoMm9fc2NhbGVkX2x1X3ByZWRbLDNdKSwgIyBTY2FsZWQgUkYgcHJlZGljdGlvbnMKICAgICAgICAgICAgICAgICAgICAgICAgICAgYXMuZGF0YS5mcmFtZShyZmgyb19ub3JtX2x1X3ByZWRbLDNdKSAgICAjIE5vcm1hbGl6ZWQgUkYgcHJlZGljdGlvbnMKKQoKY29sbmFtZXMobHVfcHJlZGljdGlvbnMpIDwtIGMoIlNTIiwgIlBvczEwd3J0c1JOQVN0YXJ0IiwgIkRpc3RUZXJtIiwgIkRpc3RhbmNlIiwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgInNhbWVTdHJhbmQiLCAiRG93bkRpc3RhbmNlIiwgInNhbWVEb3duU3RyYW5kIiwgIkNsYXNzIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiT3JpZ1JGX1AiLCJPcmlnUkZfc2NhbGVkX1AiLCJPcmlnUkZfbm9ybV9QIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiUkZfSDJPX1AiLCJSRkgyT19zY2FsZWRfUCIsIlJGSDJPX25vcm1fUCIpCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCmx1X3ByZWRpY3Rpb25zJG9yaWdQcmVkcyA8LSBpZmVsc2UoIGx1X3ByZWRpY3Rpb25zJE9yaWdSRl9QID49IDAuNSwgMSwgMCkgCmx1X3ByZWRpY3Rpb25zJG9yaWdTY2FsZWRQcmVkcyA8LSBpZmVsc2UoIGx1X3ByZWRpY3Rpb25zJE9yaWdSRl9zY2FsZWRfUCA+PSAwLjUsIDEsIDApIApsdV9wcmVkaWN0aW9ucyRvcmlnTm9ybVByZWRzIDwtIGlmZWxzZSggbHVfcHJlZGljdGlvbnMkT3JpZ1JGX25vcm1fUCA+PSAwLjUsIDEsIDApIApsdV9wcmVkaWN0aW9ucyRoMm9QcmVkcyAgPC0gaWZlbHNlKCBsdV9wcmVkaWN0aW9ucyRSRl9IMk9fUCA+PSAwLjUsIDEsIDApIApsdV9wcmVkaWN0aW9ucyRoMm9TY2FsZWRQcmVkcyAgPC0gaWZlbHNlKCBsdV9wcmVkaWN0aW9ucyRSRkgyT19zY2FsZWRfUCA+PSAwLjUsIDEsIDApIApsdV9wcmVkaWN0aW9ucyRoMm9Ob3JtUHJlZHMgIDwtIGlmZWxzZSggbHVfcHJlZGljdGlvbnMkUkZIMk9fbm9ybV9QID49IDAuNSwgMSwgMCkgIAoKbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPIDwtIE5BCmx1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdTY2FsZWQgPC0gTkEKbHVfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ05vcm0gPC0gTkEKbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkIDwtIE5BCmx1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPC0gTkEKCmZvciggaSBpbiAxOm5yb3cobHVfcHJlZGljdGlvbnMpICl7CiAgIyBCYXNlZCBvbiB0aGUgd2F5IHdlIGFyZSBmZWVkaW5nIHRoZSB2YWx1ZXMgdG8gdGhlIGNvbXBhcmVBbnN3ZXJzIGZ1bmN0aW9uLCAKICAjIFNpbWlsaWFyaXRpZXMgPSBPbmx5QSBtZWFucyB0aGF0IG9ubHkgdGhlIE9yaWcgUkYgZ290IHRoZSByaWdodCBhbnN3ZXIKICAjIFNpbWlsaWFyaXRpZXMgPSBPbmx5QiBtZWFucyB0aGF0IG9ubHkgdGhlIG90aGVyIFJGIGdvdCB0aGUgcmlnaHQgYW5zd2VyCiAgIyBBbGwgb3RoZXIgYW5zd2VycyB3aWxsIHRlbGwgdXMgd2hlcmUgYm90aCBtb2RlbHMgd2VyZSByaWdodCgiQk9USF9SSUdIVCIpLCBvcgogICMgd3JvbmcgKCJCT1RIX1dST05HIikKICBsdV9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ZzT3JpZ1NjYWxlZCA8LSBjb21wYXJlQW5zd2VycygKICAgIGx1X3ByZWRpY3Rpb25zW2ksXSRvcmlnUHJlZHMsIGx1X3ByZWRpY3Rpb25zW2ksXSRvcmlnU2NhbGVkUHJlZHMsbHVfcHJlZGljdGlvbnNbaSxdJENsYXNzICkKICBsdV9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ZzT3JpZ05vcm0gPC0gY29tcGFyZUFuc3dlcnMoCiAgICBsdV9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ByZWRzLCBsdV9wcmVkaWN0aW9uc1tpLF0kb3JpZ05vcm1QcmVkcywgIGx1X3ByZWRpY3Rpb25zW2ksXSRDbGFzcyApCiAgbHVfcHJlZGljdGlvbnNbaSxdJG9yaWdWc0gyTyA8LSBjb21wYXJlQW5zd2VycygKICAgIGx1X3ByZWRpY3Rpb25zW2ksXSRvcmlnUHJlZHMsIGx1X3ByZWRpY3Rpb25zW2ksXSRoMm9QcmVkcywgICAgICAgbHVfcHJlZGljdGlvbnNbaSxdJENsYXNzICkKICBsdV9wcmVkaWN0aW9uc1tpLF0kb3JpZ1ZzSDJPU2NhbGVkIDwtIGNvbXBhcmVBbnN3ZXJzKAogICAgbHVfcHJlZGljdGlvbnNbaSxdJG9yaWdQcmVkcywgbHVfcHJlZGljdGlvbnNbaSxdJGgyb1NjYWxlZFByZWRzLCBsdV9wcmVkaWN0aW9uc1tpLF0kQ2xhc3MgKQogIGx1X3ByZWRpY3Rpb25zW2ksXSRvcmlnVnNIMk9Ob3JtIDwtIGNvbXBhcmVBbnN3ZXJzKAogICAgbHVfcHJlZGljdGlvbnNbaSxdJG9yaWdQcmVkcywgbHVfcHJlZGljdGlvbnNbaSxdJGgyb05vcm1QcmVkcywgICBsdV9wcmVkaWN0aW9uc1tpLF0kQ2xhc3MgKQp9IAoKc3RyKGx1X3ByZWRpY3Rpb25zKQoKcmJpbmQoIGhlYWQobHVfcHJlZGljdGlvbnMsIDUpLCB0YWlsKGx1X3ByZWRpY3Rpb25zLCA1KSkKCmBgYAoKQXMgYW4gYWRkaXRpb25hbCBjaGVjaywgd2UgYWxzbyByZXZpZXcgdGhlIGNvcnJlbGF0aW9ucyBiZXR3ZWVuIHRoZSBPcmlnUkYgcHJlZGljdGlvbnMsIGFuZCB0aGUgYWx0ZXJuYXRpdmUgbW9kZWxzJyBwcmVkaWN0aW9ucyB3aXRoIHRoZSBMVSBkYXRhLgoKYGBge3J9CmNvcihsdV9wcmVkaWN0aW9uc1ssIk9yaWdSRl9QIl0sbHVfcHJlZGljdGlvbnNbLCJSRl9IMk9fUCJdKSAKY29yKGx1X3ByZWRpY3Rpb25zWywiT3JpZ1JGX1AiXSxsdV9wcmVkaWN0aW9uc1ssIk9yaWdSRl9zY2FsZWRfUCJdKQpjb3IobHVfcHJlZGljdGlvbnNbLCJPcmlnUkZfUCJdLGx1X3ByZWRpY3Rpb25zWywiT3JpZ1JGX25vcm1fUCJdKQpjb3IobHVfcHJlZGljdGlvbnNbLCJPcmlnUkZfUCJdLGx1X3ByZWRpY3Rpb25zWywiUkZIMk9fc2NhbGVkX1AiXSkgCmNvcihsdV9wcmVkaWN0aW9uc1ssIk9yaWdSRl9QIl0sbHVfcHJlZGljdGlvbnNbLCJSRkgyT19ub3JtX1AiXSkKYGBgCgoKIyMjIDExLjIgQ29tcGFyZSBPcmlnUkYgdnMgdGhlIG90aGVyIE1vZGVscyB3aXRoIHRoZSBMVSBkYXRhCkJhc2ljYWxseSwgdGhlIHNhbWUgZXhwbGFuYXRpb24gYXMgaW4gc2VjdGlvbiAxMC4yCiAgKiBBIHZhbHVlIG9mICpPbmx5QSogbWVhbnMgdGhhdCBvbmx5IHRoZSBvcmlnIFJGIGdvdCB0aGUgcmlnaHQgYW5zd2VyCiAgKiBBIHZhbHVlIG9mICpPbmx5QiogbWVhbnMgdGhhdCBvbmx5IHRoZSBvdGhlciBSRiBnb3QgdGhlIHJpZ2h0IGFuc3dlcgogICogQWxsIG90aGVyIGFuc3dlcnMgd2lsbCB0ZWxsIHVzIHdoZXJlIGJvdGggbW9kZWxzIHdlcmUgcmlnaHQgKigiQk9USF9SSUdIVCIpKiBvciB3cm9uZyAqKCJCT1RIX1dST05HIikqCgojIyMjIDExLjIuMSBDb21wYXJlIE9yaWdSRiB2cyBPcmlnUkYgU2NhbGVkIE1vZGVsIApgYGB7cn0KbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA9PSAiT25seUEiLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJPbmx5QiIsXSApCm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdTY2FsZWQgPT0gIkJPVEhfV1JPTkciLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJCT1RIX1JJR0hUIixdICkKYGBgCgojIyMjIDExLjIuMiBDb21wYXJlIE9yaWdSRiB2cyBPcmlnUkYgTm9ybSBNb2RlbCAKYGBge3J9Cm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtID09ICJPbmx5QSIsXSApCm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtID09ICJPbmx5QiIsXSApCm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtID09ICJCT1RIX1dST05HIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ05vcm0gPT0gIkJPVEhfUklHSFQiLF0gKQpgYGAKCiMjIyMgMTEuMi4zIENvbXBhcmUgT3JpZ1JGIHZzIEgyTyBNb2RlbCAKYGBge3J9Cm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiT25seUEiLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk8gPT0gIk9ubHlCIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPID09ICJCT1RIX1dST05HIixdICkKbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPID09ICJCT1RIX1JJR0hUIixdICkKYGBgCgojIyMjIDExLjIuNCBDb21wYXJlIE9yaWdSRiB2cyBIMk8gTW9kZWwgU2NhbGVkIApgYGB7cn0KbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkID09ICJPbmx5QSIsXSApCm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT1NjYWxlZCA9PSAiT25seUIiLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk9TY2FsZWQgPT0gIkJPVEhfV1JPTkciLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk9TY2FsZWQgPT0gIkJPVEhfUklHSFQiLF0gKQpgYGAKCiMjIyMgMTEuMi41IENvbXBhcmUgT3JpZ1JGIHZzIEgyTyBNb2RlbCBOb3JtYWxpemVkIApgYGB7cn0KbnJvdyggbHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPTm9ybSA9PSAiT25seUEiLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk9Ob3JtID09ICJPbmx5QiIsXSApCm5yb3coIGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIkJPVEhfV1JPTkciLF0gKQpucm93KCBsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk9Ob3JtID09ICJCT1RIX1JJR0hUIixdICkKYGBgCgojIElWKSBDT01QQVJFIE1PREVMJ1MgTUVUUklDUyAKCldoaWxlIHNlY3Rpb25zIDExLjIgYW5kIDEwLjIgc3VnZ2VzdCB0aGF0IGFsbCB0aGUgbW9kZWxzIGFyZSBiZWhhdmluZyBzaW1pbGFybHksIG1ha2luZyBhIGp1ZGdlbWVudCBzb2xlbHkgb24gYWNjdXJhY3kgY2FuIGJlIGRlY2VpdmluZy4gRm9yIHRoaXMgcmVhc29uLCB3ZSBhbHNvIGRlY2lkZWQgdG8gZGl2ZSBkZWVwZXIgaW50byB0aGUgbW9kZWwncyBwZXJmb3JtYW5jZXMsIGFuZCBkbyBhZGRpdGlvbmFsIHNpZGUgYnkgc2lkZSBjb21wYXJpc29uIG9mIG90aGVyIHBlcmZvcm1hbmNlIG1ldHJpY3MuIAoKIyMgMTIuIEdldCBQZXJmb3JtYW5jZSBNZXRyaWNzIApUaGlzICpldmFsdWF0ZURhdGEqIGZ1bmN0aW9uIHdhcyBjb3BpZWQgZGlyZWN0bHkgZnJvbSB0aGUgc291cmNlIG1hdGVyaWFscyh0aGUgW2FkZGl0aW9uYWwgbWF0ZXJpYWxdKGh0dHBzOi8vcGVlcmouY29tL2FydGljbGVzLzYzMDQvKSBpbmNsdWRlZCBpbiB0aGUgb3JpZ2luYWwgYXJ0aWNsZSksIGFzIGl0IHdhcyB0aGUgb25lIHVzZWQgdG8gZXZhbHVhdGUgdGhlIG9yaWdpbmFsIG1vZGVsLCBhbmQgd2UndmUgc2ltcGx5IGFkZGVkIGNvbW1lbnRzIGEgZmV3IGV4dHJhIGNvbW1lbnRzLiBUaGUgSDJPIGxpYnJhcnkgYWxyZWFkeSBjb250YWlucyB0b29scyB0byBvYnRhaW4gcGVyZm9ybWFuY2UgbWV0cmljcyBmcm9tIGl0cyBtb2RlbHMsIHNvIG5vIGFkZGl0aW9uYWwgZnVuY3Rpb25zIHdlcmUgbmVjZXNzYXJ5IHdpdGggdGhlIEgyTyBNTCBtb2RlbHMuICAKCmBgYHtyfQpldmFsdWF0ZURhdGEgPC0gZnVuY3Rpb24oUkYsIGRhdGEsIGxhYmVscyl7CiAgcmVxdWlyZShST0NSKQogIHJlcXVpcmUoUFJST0MpCiAgcmVxdWlyZShyYW5kb21Gb3Jlc3QpCiAgcmVzIDwtIGxpc3QoKQogIHJlcyRwcmVkRCA8LSBwcmVkaWN0KFJGLCBkYXRhLCB0eXBlID0gInByb2IiKSAgICAgICAgICAgICAgICAgICAgICAgICMgUHJlZGljdGlvbnMgYXMgR2VuZXJhdGVkIGJ5IHRoZSBtb2RlbAogIHJlcyRwcmVkIDwtIHByZWRpY3Rpb24ocmVzJHByZWREWywyXSwgYXMubG9naWNhbChsYWJlbHMpKSAgICAgICAgICAgICMgUHJlZGljdGlvbnMsIGJ1dCBjb252ZXJ0ZWQgdG8gUzQgT2JqZWN0ICAgICAgICAgIAogIHJlcyRQUiA8LSBwZXJmb3JtYW5jZShyZXMkcHJlZCwgbWVhc3VyZSA9ICJwcmVjIiwgeC5tZWFzdXJlID0gInJlYyIpICMgUHJlY2lzaW9uIFJlY2FsbCBDdXJ2ZQogIHJlcyRTUyA8LSBwZXJmb3JtYW5jZShyZXMkcHJlZCwgbWVhc3VyZT0ic2VucyIsIHgubWVhc3VyZT0ic3BlYyIpICAgICMgU2Vuc2l0aXZpdHkgdnMuIFNwZWNpZmljaXR5CiAgcmVzJGF1YyAgPC0gcGVyZm9ybWFuY2UocmVzJHByZWQsIG1lYXN1cmUgPSAiYXVjIikgICAgICAgICAgICAgICAgICAgIyBTZW5zaXRpdml0eSB2cy4gU3BlY2lmaWNpdHkgQVVDCiAgcmVzJGFjYyA8LSAgcGVyZm9ybWFuY2UocmVzJHByZWQsIG1lYXN1cmUgPSAiYWNjIikgICAgICAgICAgICAgICAgICAgIyBBY2N1cmFjeQogIHJlcyRwciA8LSBwci5jdXJ2ZShzY29yZXMuY2xhc3MwID0gcmVzJHByZWREW2xhYmVscyA9PSAxLDJdLCAgICAgICAgICMgUHJlY2lzaW9uIHZzLiBSZWNhbGwgCiAgICAgICAgICAgICAgICAgICAgIHNjb3Jlcy5jbGFzczEgPSByZXMkcHJlZERbbGFiZWxzID09IDAsMl0sIAogICAgICAgICAgICAgICAgICAgICBjdXJ2ZSA9IFQpCiAgcmV0dXJuKHJlcykKfQoKIyBPcmlnUkYgUGVyZm9ybWFuY2UgTWV0cmljcwpvcmlnUkZfc2x0Ml9wZXJmb3JtYW5jZSA8LSBldmFsdWF0ZURhdGEob3JpZ1JGLCBzbHQyZGF0YVssLThdLCBzbHQyZGF0YVssOF0pCm9yaWdSRl9zY2FsZWRfc2x0Ml9wZXJmb3JtYW5jZSA8LSBldmFsdWF0ZURhdGEob3JpZ1JGX3NjYWxlZCwgc2x0MmRhdGFfc2NhbGVkWywtOF0sIHNsdDJkYXRhX3NjYWxlZFssOF0pCm9yaWdSRl9ub3JtX3NsdDJfcGVyZm9ybWFuY2UgPC0gZXZhbHVhdGVEYXRhKG9yaWdSRl9ub3JtLCBzbHQyZGF0YV9ub3JtWywtOF0sIHNsdDJkYXRhX25vcm1bLDhdKQoKb3JpZ1JGX2x1X3BlcmZvcm1hbmNlIDwtIGV2YWx1YXRlRGF0YShvcmlnUkYsIGx1ZGF0YVssLThdLCBsdWRhdGFbLDhdKQpvcmlnUkZfc2NhbGVkX2x1X3BlcmZvcm1hbmNlIDwtIGV2YWx1YXRlRGF0YShvcmlnUkZfc2NhbGVkLCBsdWRhdGFfc2NhbGVkWywtOF0sIGx1ZGF0YV9zY2FsZWRbLDhdKQpvcmlnUkZfbm9ybV9sdV9wZXJmb3JtYW5jZSA8LSBldmFsdWF0ZURhdGEob3JpZ1JGX25vcm0sIGx1ZGF0YV9ub3JtWywtOF0sIGx1ZGF0YV9ub3JtWyw4XSkKCgojIEgyTyBSRiBQZXJmb3JtYW5jZSBNZXRyaWNzCiMgSDJPIHByb3ZpZGVzIGEgd2F5IHRvIHJldHJpZXZlIG1vc3Qgb2YgdGhlIGRlc2lyZWQgbWV0cmljcwojIGh0dHA6Ly9kb2NzLmgyby5haS9oMm8vbGF0ZXN0LXN0YWJsZS9oMm8tci9kb2NzL3JlZmVyZW5jZS9oMm8ubWV0cmljLmh0bWwKCnNsdDJkYXRhX2gybyA8LSBzbHQyZGF0YQpzbHQyZGF0YV9oMm9bLCJDbGFzcyJdIDwtIGFzLmxvZ2ljYWwoc2x0MmRhdGFfaDJvWywiQ2xhc3MiXSkgCnJmaDJvX3NsdDJfcGVyZm9ybWFuY2UgPC0gaDJvLnBlcmZvcm1hbmNlKHJmaDJvLCBuZXdkYXRhID0gYXMuaDJvKHNsdDJkYXRhX2gybykpCgpsdWRhdGFfaDJvIDwtIGx1ZGF0YQpsdWRhdGFfaDJvWywiQ2xhc3MiXSA8LSBhcy5sb2dpY2FsKGx1ZGF0YV9oMm9bLCJDbGFzcyJdKSAKcmZoMm9fbHVfcGVyZm9ybWFuY2UgPC0gaDJvLnBlcmZvcm1hbmNlKHJmaDJvLCBuZXdkYXRhID0gYXMuaDJvKGx1ZGF0YV9oMm8pKQoKc2x0MmRhdGFfaDJvX3NjYWxlZCA8LSBzbHQyZGF0YV9zY2FsZWQKc2x0MmRhdGFfaDJvX3NjYWxlZFssIkNsYXNzIl0gPC0gYXMubG9naWNhbChzbHQyZGF0YV9oMm9fc2NhbGVkWywiQ2xhc3MiXSkgCnJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfc2NhbGVkIDwtIGgyby5wZXJmb3JtYW5jZShyZmgybywgbmV3ZGF0YSA9IGFzLmgybyhzbHQyZGF0YV9oMm9fc2NhbGVkKSkKCmx1ZGF0YV9oMm9fc2NhbGVkIDwtIGx1ZGF0YV9zY2FsZWQKbHVkYXRhX2gyb19zY2FsZWRbLCJDbGFzcyJdIDwtIGFzLmxvZ2ljYWwobHVkYXRhX2gyb19zY2FsZWRbLCJDbGFzcyJdKSAKcmZoMm9fbHVfcGVyZm9ybWFuY2Vfc2NhbGVkIDwtIGgyby5wZXJmb3JtYW5jZShyZmgybywgbmV3ZGF0YSA9IGFzLmgybyhsdWRhdGFfaDJvX3NjYWxlZCkpCgpzbHQyZGF0YV9oMm9fbm9ybSA8LSBzbHQyZGF0YV9ub3JtCnNsdDJkYXRhX2gyb19ub3JtWywiQ2xhc3MiXSA8LSBhcy5sb2dpY2FsKHNsdDJkYXRhX2gyb19ub3JtWywiQ2xhc3MiXSkgCnJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfbm9ybSA8LSBoMm8ucGVyZm9ybWFuY2UocmZoMm8sIG5ld2RhdGEgPSBhcy5oMm8oc2x0MmRhdGFfaDJvX25vcm0pKQoKbHVkYXRhX2gyb19ub3JtIDwtIGx1ZGF0YV9ub3JtCmx1ZGF0YV9oMm9fbm9ybVssIkNsYXNzIl0gPC0gYXMubG9naWNhbChsdWRhdGFfaDJvX25vcm1bLCJDbGFzcyJdKSAKcmZoMm9fbHVfcGVyZm9ybWFuY2Vfbm9ybSA8LSBoMm8ucGVyZm9ybWFuY2UocmZoMm8sIG5ld2RhdGEgPSBhcy5oMm8obHVkYXRhX2gyb19ub3JtKSkKYGBgCgojIyAxMy4gQ29tcGFyZSBNZXRyaWNzICAKIyMjIDEzLjEgQWNjdXJhY3kKIyMjIyAxMy4xLjEgQWNjdXJhY3kgVGFibGUKYGBge3J9Cm1ldHJpY3NfdGFibGUgPC0gZGF0YS5mcmFtZSgiQWNjdXJhY3kiID0gMToxMikKcm93bmFtZXMobWV0cmljc190YWJsZSkgPC0gYygib3JpZ1JGX3NsdDJfcGVyZiIsICJvcmlnUkZfc2NhbGVkX3NsdDJfcGVyZiIsICJvcmlnUkZfbm9ybV9zbHQyX3BlcmYiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICJyZmgyb19zbHQyX3BlcmYiLCAicmZoMm9fc2x0Ml9wZXJmX3NjYWxlZCIsICJyZmgyb19zbHQyX3BlcmZfbm9ybSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIm9yaWdSRl9sdV9wZXJmIiwgIm9yaWdSRl9zY2FsZWRfbHVfcGVyZiIsICJvcmlnUkZfbm9ybV9sdV9wZXJmIiwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgInJmaDJvX2x1X3BlcmYiLCAgInJmaDJvX2x1X3BlcmZfc2NhbGVkIiwgICJyZmgyb19sdV9wZXJmX25vcm0iKQoKbWV0cmljc190YWJsZVsib3JpZ1JGX3NsdDJfcGVyZiIsIkFjY3VyYWN5Il0gICAgICAgIDwtIG5yb3coc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdTY2FsZWQgPT0gIkJPVEhfUklHSFQiIHwgc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJPbmx5QSIsXSApLwogIG5yb3coc2x0Ml9wcmVkaWN0aW9ucykKbWV0cmljc190YWJsZVsib3JpZ1JGX3NjYWxlZF9zbHQyX3BlcmYiLCJBY2N1cmFjeSJdIDwtIG5yb3coc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdTY2FsZWQgPT0gIkJPVEhfUklHSFQiIHwgc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnU2NhbGVkID09ICJPbmx5QiIsXSApLwogIG5yb3coc2x0Ml9wcmVkaWN0aW9ucykKbWV0cmljc190YWJsZVsib3JpZ1JGX25vcm1fc2x0Ml9wZXJmIiwiQWNjdXJhY3kiXSAgIDwtIG5yb3coc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc09yaWdOb3JtID09ICJCT1RIX1JJR0hUIiB8IHNsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ05vcm0gPT0gIk9ubHlCIixdICkvCiAgbnJvdyhzbHQyX3ByZWRpY3Rpb25zKQoKbWV0cmljc190YWJsZVsicmZoMm9fc2x0Ml9wZXJmIiwiQWNjdXJhY3kiXSAgICAgICAgIDwtIG5yb3coc2x0Ml9wcmVkaWN0aW9uc1tzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiQk9USF9SSUdIVCIgfCBzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyTyA9PSAiT25seUIiLF0gKS8KICBucm93KHNsdDJfcHJlZGljdGlvbnMpCm1ldHJpY3NfdGFibGVbInJmaDJvX3NsdDJfcGVyZl9zY2FsZWQiLCJBY2N1cmFjeSJdICA8LSBucm93KHNsdDJfcHJlZGljdGlvbnNbc2x0Ml9wcmVkaWN0aW9ucyRvcmlnVnNIMk9TY2FsZWQgPT0gIkJPVEhfUklHSFQiIHwgIHNsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkID09ICJPbmx5QiIsXSkvCiAgbnJvdyhzbHQyX3ByZWRpY3Rpb25zKQptZXRyaWNzX3RhYmxlWyJyZmgyb19zbHQyX3BlcmZfbm9ybSIsIkFjY3VyYWN5Il0gICAgPC0gbnJvdyhzbHQyX3ByZWRpY3Rpb25zW3NsdDJfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPTm9ybSA9PSAiQk9USF9SSUdIVCIgfCBzbHQyX3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIk9ubHlCIixdKS8KICBucm93KHNsdDJfcHJlZGljdGlvbnMpCgptZXRyaWNzX3RhYmxlWyJvcmlnUkZfbHVfcGVyZiIsIkFjY3VyYWN5Il0gICAgICAgICAgPC0gbnJvdyhsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk8gPT0gIkJPVEhfUklHSFQiIHwgbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPID09ICJPbmx5QSIsXSApLwogIG5yb3cobHVfcHJlZGljdGlvbnMpCm1ldHJpY3NfdGFibGVbIm9yaWdSRl9zY2FsZWRfbHVfcGVyZiIsIkFjY3VyYWN5Il0gICA8LSBucm93KGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc09yaWdTY2FsZWQgPT0gIkJPVEhfUklHSFQiIHwgbHVfcHJlZGljdGlvbnMkb3JpZ1ZzT3JpZ1NjYWxlZCA9PSAiT25seUIiLF0gKS8KICBucm93KGx1X3ByZWRpY3Rpb25zKQptZXRyaWNzX3RhYmxlWyJvcmlnUkZfbm9ybV9sdV9wZXJmIiwiQWNjdXJhY3kiXSAgICAgPC0gbnJvdyhsdV9wcmVkaWN0aW9uc1tsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnTm9ybSA9PSAiQk9USF9SSUdIVCIgfCBsdV9wcmVkaWN0aW9ucyRvcmlnVnNPcmlnTm9ybSA9PSAiT25seUIiLF0gKS8KICBucm93KGx1X3ByZWRpY3Rpb25zKQoKbWV0cmljc190YWJsZVsicmZoMm9fbHVfcGVyZiIsIkFjY3VyYWN5Il0gICAgICAgICAgIDwtIG5yb3cobHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPID09ICJCT1RIX1JJR0hUIiB8ICBsdV9wcmVkaWN0aW9ucyRvcmlnVnNIMk8gPT0gIk9ubHlCIixdICkvCiAgbnJvdyhsdV9wcmVkaWN0aW9ucykKbWV0cmljc190YWJsZVsicmZoMm9fbHVfcGVyZl9zY2FsZWQiLCJBY2N1cmFjeSJdICAgIDwtIG5yb3cobHVfcHJlZGljdGlvbnNbbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPU2NhbGVkID09ICJCT1RIX1JJR0hUIiB8IGx1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT1NjYWxlZCA9PSAiT25seUIiLF0pLwogIG5yb3cobHVfcHJlZGljdGlvbnMpCm1ldHJpY3NfdGFibGVbInJmaDJvX2x1X3BlcmZfbm9ybSIsIkFjY3VyYWN5Il0gICAgICA8LSBucm93KGx1X3ByZWRpY3Rpb25zW2x1X3ByZWRpY3Rpb25zJG9yaWdWc0gyT05vcm0gPT0gIkJPVEhfUklHSFQiIHwgbHVfcHJlZGljdGlvbnMkb3JpZ1ZzSDJPTm9ybSA9PSAiT25seUIiLF0gKS8KICBucm93KGx1X3ByZWRpY3Rpb25zKQoKbWV0cmljc190YWJsZQpgYGAKCiMjIyMgMTMuMS4yIEFjY3VyYWN5IGdyYXBocyBiYXNlZCBvbiBkaWZmZXJlbnQgdGhyZXNob2xkcygiY3V0b2ZmIikKClRoZSBMZWdlbmQgZm9yIHRoZSBncmFwaHM6CiAgKiAgQmx1ZSA9IE9yaWdSRgogICogIE1hZ2VudGEgPSBPcmlnUkYgYnV0IHNjYWxlZAogICogIEJsYWNrID0gT3JpZ1JGIGJ1dCBub3JtYWxpemVkCiAgKiAgUmVkID0gSDJPIFJGIG1vZGVsCiAgKiAgQnJvd24gPSBIMk8gUkYgbW9kZWwgd2l0aCBzY2FsZWQgZGF0YQogICogIEdyZWVuICA9IEgyTyBSRiBtb2RlbCB3aXRoIG5vcm1hbGl6ZWQgZGF0YQoKYGBge3J9CiMgU0xUMiBBY2N1cmFjeSBQbG90CnBsb3Qob3JpZ1JGX3NsdDJfcGVyZm9ybWFuY2UkYWNjLCBjb2wgPSAiYmx1ZSIsIGx3ZCA9IDUsIG1haW4gPSAiU0xUMiBBY2N1cmFjeSIpIApsaW5lcyhhcy5kb3VibGUodW5saXN0KG9yaWdSRl9zY2FsZWRfc2x0Ml9wZXJmb3JtYW5jZSRhY2NAeC52YWx1ZXMpKSwgYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX3NsdDJfcGVyZm9ybWFuY2UkYWNjQHkudmFsdWVzKSksIGNvbCA9ICJtYWdlbnRhIiwgbHdkID0gNCkgCmxpbmVzKGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fc2x0Ml9wZXJmb3JtYW5jZSRhY2NAeC52YWx1ZXMpKSwgYXMuZG91YmxlKHVubGlzdChvcmlnUkZfbm9ybV9zbHQyX3BlcmZvcm1hbmNlJGFjY0B5LnZhbHVlcykpLCBjb2wgPSAiYmxhY2siLCBsd2QgPSAxKSAKbGluZXMoaDJvLmFjY3VyYWN5KHJmaDJvX3NsdDJfcGVyZm9ybWFuY2UpLCB0eXBlID0gImwiLCBjb2wgPSAicmVkIiwgbHdkID0gMykKbGluZXMoaDJvLmFjY3VyYWN5KHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfc2NhbGVkKSwgdHlwZSA9ICJsIiwgY29sID0gImJyb3duIiwgbHdkID0gMykKbGluZXMoaDJvLmFjY3VyYWN5KHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfbm9ybSksIHR5cGUgPSAibCIsIGNvbCA9ICJncmVlbiIsIGx3ZCA9IDMpCgojIExVIEFjY3VyYWN5IFBsb3QKcGxvdChvcmlnUkZfbHVfcGVyZm9ybWFuY2UkYWNjLCBjb2wgPSAiYmx1ZSIsIGx3ZCA9IDUsIG1haW4gPSAiTFUgQWNjdXJhY3kiKQpsaW5lcyhhcy5kb3VibGUodW5saXN0KG9yaWdSRl9zY2FsZWRfbHVfcGVyZm9ybWFuY2UkYWNjQHgudmFsdWVzKSksIGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX3NjYWxlZF9sdV9wZXJmb3JtYW5jZSRhY2NAeS52YWx1ZXMpKSwgY29sID0gIm1hZ2VudGEiLCBsd2QgPSA0KSAKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfbm9ybV9sdV9wZXJmb3JtYW5jZSRhY2NAeC52YWx1ZXMpKSwgYXMuZG91YmxlKHVubGlzdChvcmlnUkZfbm9ybV9sdV9wZXJmb3JtYW5jZSRhY2NAeS52YWx1ZXMpKSwgY29sID0gImJsYWNrIiwgbHdkID0gMSkgCmxpbmVzKGgyby5hY2N1cmFjeShyZmgyb19sdV9wZXJmb3JtYW5jZSksIHR5cGUgPSAibCIsIGNvbCA9ICJyZWQiLCBsd2QgPSAzKQpsaW5lcyhoMm8uYWNjdXJhY3kocmZoMm9fbHVfcGVyZm9ybWFuY2Vfc2NhbGVkKSwgdHlwZSA9ICJsIiwgY29sID0gImJyb3duIiwgbHdkID0gMykKbGluZXMoaDJvLmFjY3VyYWN5KHJmaDJvX2x1X3BlcmZvcm1hbmNlX25vcm0pLCB0eXBlID0gImwiLCBjb2wgPSAiZ3JlZW4iLCBsd2QgPSAzKQpgYGAKCiMjIyAxMy4yIEFVQ1BSIApUaGUgTGVnZW5kIGZvciB0aGUgZ3JhcGhzOgogICogIEJsdWUgPSBPcmlnUkYKICAqICBNYWdlbnRhID0gT3JpZ1JGIGJ1dCBzY2FsZWQKICAqICBCbGFjayA9IE9yaWdSRiBidXQgbm9ybWFsaXplZAogICogIFJlZCA9IEgyTyBSRiBtb2RlbAogICogIEJyb3duID0gSDJPIFJGIG1vZGVsIHdpdGggc2NhbGVkIGRhdGEKICAqICBHcmVlbiAgPSBIMk8gUkYgbW9kZWwgd2l0aCBub3JtYWxpemVkIGRhdGEKYGBge3J9Cm1ldHJpY3NfdGFibGVbIm9yaWdSRl9zbHQyX3BlcmYiLCJBVUNQUiJdIDwtIG9yaWdSRl9zbHQyX3BlcmZvcm1hbmNlJHByJGF1Yy5pbnRlZ3JhbAptZXRyaWNzX3RhYmxlWyJvcmlnUkZfc2NhbGVkX3NsdDJfcGVyZiIsIkFVQ1BSIl0gPC0gb3JpZ1JGX3NjYWxlZF9zbHQyX3BlcmZvcm1hbmNlJHByJGF1Yy5pbnRlZ3JhbAptZXRyaWNzX3RhYmxlWyJvcmlnUkZfbm9ybV9zbHQyX3BlcmYiLCJBVUNQUiJdIDwtIG9yaWdSRl9ub3JtX3NsdDJfcGVyZm9ybWFuY2UkcHIkYXVjLmludGVncmFsCm1ldHJpY3NfdGFibGVbInJmaDJvX3NsdDJfcGVyZiIsICJBVUNQUiJdIDwtIGgyby5hdWNwcihyZmgyb19zbHQyX3BlcmZvcm1hbmNlKQptZXRyaWNzX3RhYmxlWyJyZmgyb19zbHQyX3BlcmZfc2NhbGVkIiwgIkFVQ1BSIl0gPC0gaDJvLmF1Y3ByKHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfc2NhbGVkKQptZXRyaWNzX3RhYmxlWyJyZmgyb19zbHQyX3BlcmZfbm9ybSIsICJBVUNQUiJdIDwtIGgyby5hdWNwcihyZmgyb19zbHQyX3BlcmZvcm1hbmNlX25vcm0pCgptZXRyaWNzX3RhYmxlWyJvcmlnUkZfbHVfcGVyZiIsICAiQVVDUFIiXSA8LSBvcmlnUkZfbHVfcGVyZm9ybWFuY2UkcHIkYXVjLmludGVncmFsCm1ldHJpY3NfdGFibGVbIm9yaWdSRl9zY2FsZWRfbHVfcGVyZiIsICAiQVVDUFIiXSA8LSBvcmlnUkZfc2NhbGVkX2x1X3BlcmZvcm1hbmNlJHByJGF1Yy5pbnRlZ3JhbAptZXRyaWNzX3RhYmxlWyJvcmlnUkZfbm9ybV9sdV9wZXJmIiwgICJBVUNQUiJdIDwtIG9yaWdSRl9ub3JtX2x1X3BlcmZvcm1hbmNlJHByJGF1Yy5pbnRlZ3JhbAptZXRyaWNzX3RhYmxlWyJyZmgyb19sdV9wZXJmIiwgICJBVUNQUiJdIDwtIGgyby5hdWNwcihyZmgyb19sdV9wZXJmb3JtYW5jZSkKbWV0cmljc190YWJsZVsicmZoMm9fbHVfcGVyZl9zY2FsZWQiLCAiQVVDUFIiXSA8LSBoMm8uYXVjcHIocmZoMm9fbHVfcGVyZm9ybWFuY2Vfc2NhbGVkKQptZXRyaWNzX3RhYmxlWyJyZmgyb19sdV9wZXJmX25vcm0iLCAiQVVDUFIiXSA8LSBoMm8uYXVjcHIocmZoMm9fbHVfcGVyZm9ybWFuY2Vfbm9ybSkKCm1ldHJpY3NfdGFibGUKCnBsb3Qob3JpZ1JGX3NsdDJfcGVyZm9ybWFuY2UkUFIsICBjb2wgPSAiYmx1ZSIsIGx3ZCA9IDUsIG1haW4gPSAiU0xUMiBQUiBDdXJ2ZSIgKQpsaW5lcyhhcy5kb3VibGUodW5saXN0KG9yaWdSRl9zY2FsZWRfc2x0Ml9wZXJmb3JtYW5jZSRQUkB4LnZhbHVlcykpLCBhcy5kb3VibGUodW5saXN0KG9yaWdSRl9zY2FsZWRfc2x0Ml9wZXJmb3JtYW5jZSRQUkB5LnZhbHVlcykpLCAgY29sID0gIm1hZ2VudGEiLCBsd2QgPSA0ICkKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfbm9ybV9zbHQyX3BlcmZvcm1hbmNlJFBSQHgudmFsdWVzKSksIGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fc2x0Ml9wZXJmb3JtYW5jZSRQUkB5LnZhbHVlcykpLCAgY29sID0gImJsYWNrIiwgbHdkID0gMiApCgpsaW5lcyh4ID0gaDJvLnJlY2FsbChyZmgyb19zbHQyX3BlcmZvcm1hbmNlKVssInRwciJdLAogICAgICB5ID0gaDJvLnByZWNpc2lvbihyZmgyb19zbHQyX3BlcmZvcm1hbmNlKVssInByZWNpc2lvbiJdLAogICAgICBjb2wgPSAicmVkIiwgdHlwZSA9ICJsIiwgbHdkID0gMwopCmxpbmVzKHggPSBoMm8ucmVjYWxsKHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfc2NhbGVkKVssInRwciJdLAogICAgICB5ID0gaDJvLnByZWNpc2lvbihyZmgyb19zbHQyX3BlcmZvcm1hbmNlX3NjYWxlZClbLCJwcmVjaXNpb24iXSwKICAgICAgY29sID0gImJyb3duIiwgdHlwZSA9ICJsIiwgbHdkID0gMwopCmxpbmVzKHggPSBoMm8ucmVjYWxsKHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfbm9ybSlbLCJ0cHIiXSwKICAgICAgeSA9IGgyby5wcmVjaXNpb24ocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZV9ub3JtKVssInByZWNpc2lvbiJdLAogICAgICBjb2wgPSAiZ3JlZW4iLCB0eXBlID0gImwiLCBsd2QgPSAzCikKCgpwbG90KG9yaWdSRl9sdV9wZXJmb3JtYW5jZSRQUiwgIGNvbCA9ICJibHVlIiwgbHdkID0gNSwgbWFpbiA9ICJMVSBQUiBDdXJ2ZSIgKQpsaW5lcyhhcy5kb3VibGUodW5saXN0KG9yaWdSRl9zY2FsZWRfbHVfcGVyZm9ybWFuY2UkUFJAeC52YWx1ZXMpKSwgYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX2x1X3BlcmZvcm1hbmNlJFBSQHkudmFsdWVzKSksICBjb2wgPSAibWFnZW50YSIsIGx3ZCA9IDQgKQpsaW5lcyhhcy5kb3VibGUodW5saXN0KG9yaWdSRl9ub3JtX2x1X3BlcmZvcm1hbmNlJFBSQHgudmFsdWVzKSksIGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fbHVfcGVyZm9ybWFuY2UkUFJAeS52YWx1ZXMpKSwgIGNvbCA9ICJibGFjayIsIGx3ZCA9IDIgKQpsaW5lcyh4ID0gaDJvLnJlY2FsbChyZmgyb19sdV9wZXJmb3JtYW5jZSlbLCJ0cHIiXSwKICAgICAgeSA9IGgyby5wcmVjaXNpb24ocmZoMm9fbHVfcGVyZm9ybWFuY2UpWywicHJlY2lzaW9uIl0sCiAgICAgIGNvbCA9ICJyZWQiLCB0eXBlID0gImwiLCBsd2QgPSAzCikKbGluZXMoeCA9IGgyby5yZWNhbGwocmZoMm9fbHVfcGVyZm9ybWFuY2Vfc2NhbGVkKVssInRwciJdLAogICAgICB5ID0gaDJvLnByZWNpc2lvbihyZmgyb19sdV9wZXJmb3JtYW5jZV9zY2FsZWQpWywicHJlY2lzaW9uIl0sCiAgICAgIGNvbCA9ICJicm93biIsIHR5cGUgPSAibCIsIGx3ZCA9IDMKKQpsaW5lcyh4ID0gaDJvLnJlY2FsbChyZmgyb19sdV9wZXJmb3JtYW5jZV9ub3JtKVssInRwciJdLAogICAgICB5ID0gaDJvLnByZWNpc2lvbihyZmgyb19sdV9wZXJmb3JtYW5jZV9ub3JtKVssInByZWNpc2lvbiJdLAogICAgICBjb2wgPSAiZ3JlZW4iLCB0eXBlID0gImwiLCBsd2QgPSAzCikKYGBgCgojIyMgMTMuMyBTZW5zaXRpdnkgdnMgU3BlY2lmaWNpdHkgR3JhcGggClRoZSBMZWdlbmQgZm9yIHRoZSBncmFwaHM6CiAgKiAgQmx1ZSA9IE9yaWdSRgogICogIE1hZ2VudGEgPSBPcmlnUkYgYnV0IHNjYWxlZAogICogIEJsYWNrID0gT3JpZ1JGIGJ1dCBub3JtYWxpemVkCiAgKiAgUmVkID0gSDJPIFJGIG1vZGVsCiAgKiAgQnJvd24gPSBIMk8gUkYgbW9kZWwgd2l0aCBzY2FsZWQgZGF0YQogICogIEdyZWVuICA9IEgyTyBSRiBtb2RlbCB3aXRoIG5vcm1hbGl6ZWQgZGF0YQpgYGB7cn0KcGxvdChvcmlnUkZfc2x0Ml9wZXJmb3JtYW5jZSRTUywgY29sID0gImJsdWUiLCBsd2QgPSA1LCBtYWluID0gIlNMVDIgU2Vuc2l0aXZpdHkgdnMgU3BlY2lmaWNpdHkiKQpsaW5lcyhhcy5kb3VibGUodW5saXN0KG9yaWdSRl9zY2FsZWRfc2x0Ml9wZXJmb3JtYW5jZSRTU0B4LnZhbHVlcykpLCBhcy5kb3VibGUodW5saXN0KG9yaWdSRl9zY2FsZWRfc2x0Ml9wZXJmb3JtYW5jZSRTU0B5LnZhbHVlcykpLCAgY29sID0gIm1hZ2VudGEiLCBsd2QgPSA0ICkKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfbm9ybV9zbHQyX3BlcmZvcm1hbmNlJFNTQHgudmFsdWVzKSksIGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX25vcm1fc2x0Ml9wZXJmb3JtYW5jZSRTU0B5LnZhbHVlcykpLCAgY29sID0gImJsYWNrIiwgbHdkID0gMiApCgpsaW5lcyh4ID0gaDJvLnNwZWNpZmljaXR5KHJmaDJvX3NsdDJfcGVyZm9ybWFuY2UpWywidG5yIl0sCiAgICAgeSA9IGgyby5zZW5zaXRpdml0eShyZmgyb19zbHQyX3BlcmZvcm1hbmNlKVssInRwciJdLAogICAgIGNvbCA9ICJyZWQiLCB0eXBlID0gImwiLCBsd2QgPSAzCiAgICAgKQoKbGluZXMoeCA9IGgyby5zcGVjaWZpY2l0eShyZmgyb19zbHQyX3BlcmZvcm1hbmNlX3NjYWxlZClbLCJ0bnIiXSwKICAgICAgeSA9IGgyby5zZW5zaXRpdml0eShyZmgyb19zbHQyX3BlcmZvcm1hbmNlX3NjYWxlZClbLCJ0cHIiXSwKICAgICAgY29sID0gImJyb3duIiwgdHlwZSA9ICJsIiwgbHdkID0gMwopCmxpbmVzKHggPSBoMm8uc3BlY2lmaWNpdHkocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZV9ub3JtKVssInRuciJdLAogICAgICB5ID0gaDJvLnNlbnNpdGl2aXR5KHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfbm9ybSlbLCJ0cHIiXSwKICAgICAgY29sID0gImdyZWVuIiwgdHlwZSA9ICJsIiwgbHdkID0gMwopCgpwbG90KG9yaWdSRl9sdV9wZXJmb3JtYW5jZSRTUywgY29sID0gImJsdWUiLCBsd2QgPSA1LCBtYWluID0gIkxVIFNlbnNpdGl2aXR5IHZzIFNwZWNpZmljaXR5IikKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfc2NhbGVkX2x1X3BlcmZvcm1hbmNlJFNTQHgudmFsdWVzKSksIGFzLmRvdWJsZSh1bmxpc3Qob3JpZ1JGX3NjYWxlZF9sdV9wZXJmb3JtYW5jZSRTU0B5LnZhbHVlcykpLCAgY29sID0gIm1hZ2VudGEiLCBsd2QgPSA0ICkKbGluZXMoYXMuZG91YmxlKHVubGlzdChvcmlnUkZfbm9ybV9sdV9wZXJmb3JtYW5jZSRTU0B4LnZhbHVlcykpLCBhcy5kb3VibGUodW5saXN0KG9yaWdSRl9ub3JtX2x1X3BlcmZvcm1hbmNlJFNTQHkudmFsdWVzKSksICBjb2wgPSAiYmxhY2siLCBsd2QgPSAyICkKbGluZXMoeCA9IGgyby5zcGVjaWZpY2l0eShyZmgyb19sdV9wZXJmb3JtYW5jZSlbLCJ0bnIiXSwKICAgICAgeSA9IGgyby5zZW5zaXRpdml0eShyZmgyb19sdV9wZXJmb3JtYW5jZSlbLCJ0cHIiXSwKICAgICAgY29sID0gInJlZCIsIHR5cGUgPSAibCIsIGx3ZCA9IDMKKQoKbGluZXMoeCA9IGgyby5zcGVjaWZpY2l0eShyZmgyb19sdV9wZXJmb3JtYW5jZV9zY2FsZWQpWywidG5yIl0sCiAgICAgIHkgPSBoMm8uc2Vuc2l0aXZpdHkocmZoMm9fbHVfcGVyZm9ybWFuY2Vfc2NhbGVkKVssInRwciJdLAogICAgICBjb2wgPSAiYnJvd24iLCB0eXBlID0gImwiLCBsd2QgPSAzCikKCmxpbmVzKHggPSBoMm8uc3BlY2lmaWNpdHkocmZoMm9fbHVfcGVyZm9ybWFuY2Vfbm9ybSlbLCJ0bnIiXSwKICAgICAgeSA9IGgyby5zZW5zaXRpdml0eShyZmgyb19sdV9wZXJmb3JtYW5jZV9ub3JtKVssInRwciJdLAogICAgICBjb2wgPSAiZ3JlZW4iLCB0eXBlID0gImwiLCBsd2QgPSAzCikKYGBgCgojIyMgMTMuNCBPdGhlciBIMk8gTWV0cmljcyAKU29tZSBhZGRpdGlvbmFsIGluZm9ybWF0aW9uIHRoYXQncyBlYXNpbHkgb2J0YWluYWJsZSBmcm9tIHRoZSBIMk8gbGlicmFyeS4gCgpgYGB7cn0KaDJvLmNvbmZ1c2lvbk1hdHJpeChyZmgyb19zbHQyX3BlcmZvcm1hbmNlKQpoMm8uY29uZnVzaW9uTWF0cml4KHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfc2NhbGVkKQpoMm8uY29uZnVzaW9uTWF0cml4KHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfbm9ybSkKCnBsb3QoaDJvLkYxKHJmaDJvX3NsdDJfcGVyZm9ybWFuY2UpLCBjb2wgPSAiYmx1ZSIsIG1haW4gPSAiRjEgUGVyZm9ybWFuY2UiKQpsaW5lcyhoMm8uRjEocmZoMm9fc2x0Ml9wZXJmb3JtYW5jZV9zY2FsZWQpLCBjb2wgPSAicmVkIikKbGluZXMoaDJvLkYxKHJmaDJvX3NsdDJfcGVyZm9ybWFuY2Vfbm9ybSksIGNvbCA9ICJvcmFuZ2UiKQoKcGxvdChyZmgyb19zbHQyX3BlcmZvcm1hbmNlLCAjIFJFTUlOREVSOiBUUFIgPSBTZW5zaXRpdml0eSwgRlBSID0gKDEgLSBzcGVjaWZpY2l0eSkKICAgICB0eXBlID0gInJvYyIsIAogICAgIGNvbCA9ICJyZWQiLAogICAgIGNleCA9IDAuMiwKICAgICBwY2ggPSAxMAopCgpoMm8uY29uZnVzaW9uTWF0cml4KHJmaDJvX2x1X3BlcmZvcm1hbmNlKQpwbG90KGgyby5GMShyZmgyb19sdV9wZXJmb3JtYW5jZSkpCnBsb3QocmZoMm9fbHVfcGVyZm9ybWFuY2UsCiAgICAgdHlwZSA9ICJyb2MiLCAKICAgICBjb2wgPSAicmVkIiwKICAgICBjZXggPSAwLjIsCiAgICAgcGNoID0gMTAKKQpgYGAKCiMgVikgUkZFL1BEUHMKClJhbmRvbSBGb3Jlc3QgRXhwbGFpbmVyKFJGRSkgaXMgYSBnbG9iYWwgaW50ZXJwcmV0YWJpbGl0eSBtb2RlbCB0aGF0IHVsdGltYXRlbHkgYWxsb3dzIHlvdSB0byBiZXR0ZXIgdW5kZXJzdGFuZCB5b3VyIFJGIG1vZGVsIGFuZCBjcmVhdGUgdmlzdWFsbHkgYXR0cmFjdGl2ZSBhbmQgdXNlZnVsIFBEUCBwbG90cy4gClRoZSBtZXRob2RvbG9neSBmb3IgYmVoaW5kIFJGRSBpcyBzaW1wbGU6IGlkZW50aWZ5IHRoZSB0b3AgbW9zdCBpbXBvcnRhbnQgZmVhdHVyZXMsIHRoZWlyIGludGVyYWN0aW9ucywgYW5kIHRoZW4gZ3JhcGggdGhlc2UgaW50ZXJhY3Rpb25zIGluIHdoYXQgaXMgZXNzZW50aWFsbHkgYSAyRCBQRFAgcGxvdC4gClVuZm9ydHVuYXRlbHksIHdoZW4gdHJ5aW5nIHRvIHVzZSBSRkUgd2l0aGluIHRoZSBub3RlYm9vaywgdGhlIG91dHB1dHMgZ2VuZXJhdGVkIGVycm9ycyBhbmQgdGhlIHBsb3RzIHdlcmUgc2ltcGx5IG5vdCBzaG93aW5nLiAKVG8gZ2V0IGFyb3VuZCB0aGlzIHByb2JsZW0sIHdlIHdpbGwgc2ltcGx5IGFkZCB0aGUgZ3JhcGhzIGFzIGdlbmVyYXRlZCBieSB0aGUgUkZFIGxpYnJhcnkuIApUaGUgY29kZSB0aGF0IGdlbmVyYXRlZCB0aGVzZSBncmFwaHMgYmUgZm91bmQgaW4gW2hlcmVdKC4vUkZFX3NSTkEuUiksIGFuZCB0aGUgb3V0cHV0IGh0bWwgZ2VuZXJhdGVkIGJ5IHRoZSAqZXhwbGFpbl9mb3Jlc3QoKSogZnVuY3Rpb24gY2FuIGFsc28gYmUgZm91bmQgWyIuL1lvdXJfZm9yZXN0X2V4cGxhaW5lZC5odG1sIl0oLi9Zb3VyX2ZvcmVzdF9leHBsYWluZWQuaHRtbCkuIApLZWVwIGluIG1pbmQgdGhhdCB0aGUgSFRNTCBmaWxlIGlzIGF1dG9tYXRpY2FsbHkgZ2VuZXJhdGVkIGJ5IHRoZSBsaWJyYXJ5IGFuZCBkb2VzIG5vdCBpbmNsdWRlIGFsbCB0aGUgcGxvdHMgd2UgZ2VuZXJhdGVkIChpdCBzZWVtcyB0byBvbmx5IHNob3cgd2hhdCBpdCBjb25zaWRlcnMgdGhlIG1haW4gcGxvdHMpLiAKU2luY2UgdGhlIHlvdSBzdGlsbCBoYXZlIHRvIGdlbmVyYXRlIHRoZSBwbG90cyB5b3Vyc2VsZiwgdGhlIEhUTUwgZmlsZSBpcyBhIGNvb2wgYm9udXMsIGJ1dCBub3QgYSBuZWNlc3NhcnkuIAoKIyMgMTQuIFZhcmlhYmxlIEltcG9ydGFuY2UgTWVhc3VyZXMKIyMjIDE0LjEgRGlzdHJpYnV0aW9uIG9mIE1pbmltYWwgRGVwdGggYW5kIGl0cyBNZWFuCgohW0Rpc3RyaWJ1dGlvbiBvZiBtaW5pbWFsIGRlcHRoIGFuZCBpdHMgbWVhbl0oLi9SZXN1bHRzL3JmZV9taW5fZGVwdGhfZGlzdHJpYnV0aW9uLnBuZykKCiMjIyAxNC4yIEltcG9ydGFuY2UgTWVhc3VyZXMgVGFibGUgCgpgYGB7cn0KcmZlX2ltcG9ydGFuY2VfZnJhbWUgPC0gcmVhZC5jc3YoIi4vUmVzdWx0cy9SRkVfaW1wb3J0YW5jZV9mcmFtZV9vcmlnUkYuY3N2Iiwgc2VwID0gIiwiLCBoZWFkZXIgPSBUUlVFKQpyZmVfaW1wb3J0YW5jZV9mcmFtZSAKYGBgCgojIyMgMTQuMyBNdWx0aXdheSBJbXBvcnRhbmNlIFBsb3RzCiMjIyMgMTQuMy4xIFRpbWVzIGEgcm9vdCB2cyBNZWFuIG1pbiBkZXB0aCB2cyBOdW1iZXIgb2YgTm9kZXMKCiFbVGltZXMgYSByb290IHZzIE1lYW4gbWluIGRlcHRoIHZzIE51bWJlciBvZiBOb2Rlc10oLi9SZXN1bHRzL3JmZV9tdWx0aXdheV9pbXBfcGxvdF8xLnBuZykKCiMjIyMgMTQuMy4yIEFjY3VyYWN5IERlY3JlYXNlIHZzLiBHaW5pIERlY3JlYXNlIHZzLiBUaW1lcyBhIFJvb3QKCiFbQWNjdXJhY3kgRGVjcmVhc2UgdnMuIEdpbmkgRGVjcmVhc2UgdnMuIFRpbWVzIGEgUm9vdF0oLi9SZXN1bHRzL3JmZV9tdWx0aXdheV9pbXBfcGxvdF8yLnBuZykKCiMjIDE1LiBHR3BhaXJzIENvbXBhcmlzb25zCgpUaGUgZm9sbG93aW5nIGdyYXBocyBhcmUgc2ltcGx5IGhlbHBmdWwgaW4gaWRlbnRpZnlpbmcgZmVhdHVyZSBpbXBvcnRhbmNlCiFbZ2dwYWlycyBmdWxsXSguL1Jlc3VsdHMvcmZlX2ltcF9nZ3BhaXJzX2Z1bGwucG5nKQoKIVtnZ3BhcmlzIG1haW4gZmVhdHVyZXNdKC4vUmVzdWx0cy9yZmVfaW1wX2dncGFpcnNfbWFpbl9wYWlycy5wbmcpCgohW2dncGFpcnMgaW1wb3J0YW5jZSByYW5raW5nc10oLi9SZXN1bHRzL3JmZV9pbXBfcmFua2luZ3MucG5nKQoKIyMgMTYuIEZlYXR1cmUgSW50ZXJhY3Rpb25zCgojIyMgMTYuMSBUb3AgMzAgRmVhdHVyZSBJbnRlcmFjdGlvbnMKCiFbVG9wIDMwIEZlYXR1cmUgSW50ZXJhY3Rpb25zXSguL1Jlc3VsdHMvcmZlX21pbl9kZXB0aF9pbnRlcmFjdGlvbnMucG5nKSAKCkZyb20gdGhpcyBncmFwaCwgd2UgZ2F0aGVyIHRoYXQgdGhlIHRvcCA1IGZlYXR1cmUgaW50ZXJhY3Rpb25zIGFyZToKICAxLiBEb3duRGlzdGFuY2U6U1MKICAxLiBEaXN0YW5jZTpTUwogIDEuIERvd25EaXN0YW5jZTpQb3MxMHdydHNSTkFTdGFydCwgCiAgMS4gRGlzdFRlcm06U1MsIAogIDEuIERpc3RhbmNlOlBvczEwd3J0c1JOQVN0YXJ0CkV2ZW4gdGhvdWdoIERpc3RhbmNlIGFuZCBEb3duRGlzdGFuY2Ugd2VyZSBjb25zaWRlcmVkIHRoZSBtb3N0IGltcG9ydGFudCBmZWF0dXJlcywgdGhlaXIgaW50ZXJhY3Rpb25zIGRpZCBub3QgYXBwZWFyIHRvIGJlIGFzIHNpZ25pZmljYW50IGFzIHRoZSBvdGhlcnMKCgoKIyMjIDE2LjIgRG93bkRpc3RhbmNlIHZzLiBTUwoKIVtEb3duRGlzdGFuY2UgdnMuIFNTXSguL1Jlc3VsdHMvcmZlX0Rvd25EaXN0YW5jZV92c19TUy5wbmcpCiogVGhlIGhpZ2hlcihjbG9zZXIgdG8gMCkgdGhlIFNTLCB0aGUgaGlnaGVyIHRoZSBjb25maWRlbmNlIGluIHRoZSBwcmVkaWN0aW9uCgojIyMgMTYuMyBEaXN0YW5jZSB2cy4gU1MKCiFbRGlzdGFuY2UgdnMuIFNTXSguL1Jlc3VsdHMvcmZlX0Rpc3RhbmNlX3ZzX1NTLnBuZykKKiBkaXN0YW5jZXMgZ3JlYXRlciB0aGFuIC04NTAgPT4gaGlnaGVyIGNoYW5jZXMgb2YgaGF2aW5nIGEgYm9uYSBmaWRlIHNSTkEKKiBTUyBoaWdoZXIgdGhhbiAtMTAgPT4gTG93ZXIgdGhlIHByb2Igb2YgaGF2aW5nIGEgYm9uYSBmaWRlIHNSTkEKCiMjIyAxNi40IERvd25EaXN0YW5jZSB2cy4gUG9zMTB3cnRzUk5BU3RhcnQKCiFbRG93bkRpc3RhbmNlIHZzLiBQb3MxMHdydHNSTkFTdGFydF0oLi9SZXN1bHRzL3JmZV9Eb3duRGlzdGFuY2VfdnNfUG9zMTB3cnRzUk5BU3RhcnQucG5nKSAKKiBObyBjbGVhciBkaXNlcm5hYmxlIHRha2Vhd2F5CgojIyMgMTYuNSBEaXN0VGVybSB2cy4gU1MKCiFbRGlzdFRlcm0gdnMuIFNTXSguL1Jlc3VsdHMvcmZlX0Rpc3RUZXJtX3ZzX1NTLnBuZykKKiBTUyBvZiAwIHNlZW1zIHRvIHZpcnR1YWxseSBndWFyYW50ZWUgdGhhdCB3ZSBkb24ndCBoYXZlIGEgYm9uYSBmaWRlIHNSTkEKCiMjIyAxNi42IERpc3RhbmNlIHZzLiBQb3MxMHdydHNSTkFTdGFydAoKIVtEaXN0YW5jZSB2cy4gUG9zMTB3cnRzUk5BU3RhcnRdKC4vUmVzdWx0cy9yZmVfRGlzdGFuY2VfdnNfUG9zMTB3cnRzUk5BU3RhcnQucG5nKSAKKiBIYXZpbmcgYSBQb3MxMHdydHNSTkFTdGFydCBuZWFyIDAgb3Igc2xpZ2h0bHkgbG93ZXIgc2VlbXMgdG8gbWFrZSBwcmVkaWN0aW9ucyBmdXp6eSAobmVhciAwLjUpCgoKIyMjIDE2LjcgRGlzdGFuY2UgdnMuIERvd25EaXN0YW5jZQoKIVtEaXN0YW5jZSB2cy4gRG93bkRpc3RhbmNlXSguL1Jlc3VsdHMvcmZlX0Rpc3RhbmNlX3ZzX0Rvd25EaXN0YW5jZS5wbmcpIAoqIFRoZXNlIGFyZSB0aGUgMiB0b3AgZmVhdHVyZXMsIGJ1dCB0aGVpciBpbnRlcmFjdGlvbnMgYXJlIG5vdCBhbW9uZyB0aGUgdG9wIG9uZXMKKiBXaGVuIGJvdGggZmVhdHVyZXMgYXJlIGNsb3NlIHRvIDAsIHRoZSBjaGFuY2VzIG9mIGhhdmluZyBhIGJvbmEgZmlkZSBzUk5BIGdyZWF0bHkgaW5jcmVhc2UKZGV2Lm9mZigpCgojIyMgMTYuOCBEb3duRGlzdGFuY2UgdnMuIERpc3RUZXJtCgpCYXNlZCBvbiB0aGUgcHJldmlvdXMgcGxvdHMsIGdlbmVyYXRpbmcgdGhlIGZvbGxvd2luZyBwbG90cyBzZWVtIGxpa2UgYSBzbWFydCB0aGluZyB0byBkbwohW0Rvd25EaXN0YW5jZSB2cy4gRGlzdFRlcm1dKC4vUmVzdWx0cy9yZmVfRG93bkRpc3RhbmNlX3ZzX0Rpc3RUZXJtLnBuZykKKiBoYXZpbmcgYSBEaXN0VGVybSBsZXNzIHRoYW4gNTAsIG9yIGNsb3NlIHRvIDAsIGluY3JlYXNlcyB0aGUgY2hhbmNlcyBvZiBoYXZpbmcgYSBib25hIGZpZGUgc1JOQQoKIyMjIDE2LjkgRGlzdGFuY2UgdnMuIERpc3RUZXJtCgohW0Rpc3RhbmNlIHZzLiBEaXN0VGVybV0oLi9SZXN1bHRzL3JmZV9EaXN0YW5jZV92c19EaXN0VGVybS5wbmcpCgoqIGhhdmluZyBhIERpc3RhbmNlIGNsb3NlIHRvIDAgaW5jcmVhc2VzIHRoZSBjaGFuY2VzIG9mIGhhdmluZyBhIGJvbmEgZmlkZSBzUk5BCgojIyMgMTYuMTAgc2FtZVN0cmFuZCB2cyBzYW1lRG93blN0cmFuZAoKIVtzYW1lU3RyYW5kIHZzIHNhbWVEb3duU3RyYW5kXSguL1Jlc3VsdHMvcmZlX3NhbWVTdHJhbmRfdnNfc2FtZURvd25TdHJhbmQucG5nKQoKKiBzYW1lU3RyYW5kIGFuZCBzYW1lRG93blN0cmFuZCB3ZXJlIGNvbnNpZGVyZWQgdGhlIGxlYXN0IHJlbGV2YW50L2ltcG9ydGFudCBmZWF0dXJlcy4gSG93ZXZlciwgdGhlIG92ZXJhbGwgdG9uZSBvZiB0aGUgZ3JhcGggc3VnZ2VzdHMgdGhhdCBoYXZpbmcgdGhlc2UgdmFyaWFibGVzIGhlbHBzIGJvb3N0aW5nIHRoZSBtb2RlbCdzIGNvbmZpZGVuY2UgaW4gYW55IGdpdmVuIHByZWRpY3Rpb24uIChpZS4gY29tcGFyZSB0aGUgdG9uZXMgb2YgdGhlIHByZXZpb3VzIGdyYXBoIHRvIHRoaXMgb25lLCBhbmQgaXQncyBvYnZpb3VzIHRoZSBtb2RlbCBoYXMgbW9yZSAiY29uZmlkZW5jZSIgd2l0aCBpdHMgb3ZlcmFsbCBkZWNpc2lvbnMgaW4gdGhpcyBsYXRlciBncmFwaC4pCgoKIyMgMTcuIFJGRSBDb25jbHVzaW9ucwoKVGhlcmUgYXJlIG1hbnkgY29uY2x1c2lvbnMgdGhhdCBtYXkgYmUgZ2F0aGVyZWQgb3Igb2J0YWluZWQgZnJvbSB0aGUgcGxvdHMgYWJvdmUsIGJ1dCB0aGUgbWFpbiB0YWtlYXdheXMgY2FuIGJlIHN1bW1hcml6ZWQgYXMgZm9sbG93czoKICAqIFRoZSAgY2xvc2VyIHRoZSBTUyBpcyB0byAwLCB0aGUgaGlnaGVyIHRoZSBtb2RlbCdzIGNvbmZpZGVuY2UgaW4gaXRzIGRlY2lzaW9uCiAgKiAqKkhhdmluZyBhIERpc3RhbmNlIGJldHdlZW4gLTg1MCBhbmQgMCBncmVhdGx5IGluY3JlYXNlcyB0aGUgY2hhbmNlcyBvZiBoYXZpbmcgYW4gc1JOQSoqCiAgKiBBbiBTUyBvZiAtMTIgb3IgaGlnaGVyIHdpbGwgYWxtb3N0IGd1YXJhbnRlZSBhIG5vbiBib25hIGZpZGUgc1JOQQogICogVGhlcmUgaXMgYSBzd2VldCBzcG90IGZvciBmaW5kaW5nIGJvbmEgZmlkZSBzUk5BcyBpZiB0aGUgRG93bkRpc3RhbmNlIGlzIGxlc3MgdGhhbiA1MDAgYW5kIHRoZSBEaXN0YW5jZSBpcyBsZXNzIHRoYW4gODAwCiAgKiAqKkhhdmluZyBhIERvd25EaXN0YW5jZSBsZXNzIHRoYW4gNjAgYWxzbyBncmVhdGx5IGluY3JlYXNlcyB0aGUgY2hhbmNlcyBvZiBoYXZpbmcgYW4gc1JOQSoqCiAgKiBXaGlsZSBzYW1lU3RyYW5kIGFuZCBzYW1lRG93blN0cmFuZCB3ZXJlIGNvbnNpZGVyZWQgdGhlIGxlYXN0IGltcG9ydGFudCBmZWF0dXJlcywgdGhleSBzZWVtIHRvIGNvbnRyaWJ1dGUgaW4gaW5jcmVhc2luZyB0aGUgbW9kZWwncyBjb25maWRlbmNlIChhdCBsZWFzdCBpbiBjbGFzc2lmeWluZyBzb21ldGhpbmcgYXMgbm90IGJlaW5nIGFuIHNSTkEsIGFzIHRoZSBncmFwaCBzZWVtcyB0byBjb250YWluIG1vc3RseSBzb2xpZCBibHVlIGNvbG9ycykKICAKKk5PVEUvRElTQ0xBSU1FUjoqIFRoZSBudW1iZXJzIGhlcmUgYXJlIGh1bWFuIGFwcHJveGltYXRpb25zIGJhc2VkIHNvbGVseSBvbiBsb29raW5nIGF0IHRoZSBncmFwaHMgZ2VuZXJhdGVkIGJ5IFJGRSwgYW5kIFJGRSB1c2VzIG9ubHkgdGhlIHRyYWluaW5nIGRhdGEgdG8gZ2VuZXJhdGUgaXRzIGdyYXBocy4gCkluIG90aGVyIHdvcmRzLCB3aGlsZSB0aGUgZ29hbCBvZiB0aGUgUkYgaXMgdG8gaGF2ZSBhIGdlbmVyYWwgbW9kZWwgYXBwbGljYWJsZSB0byBhbnkgcG9zc2libGUgYmFjdGVyaWFsIHNSTkEsIHRoZXJlIGlzIG5vIGd1YXJhbnRlZSB0aGF0IHRoZXNlIGNvbmNsdXNpb25zIGFwcGx5IHRvIGV2ZXJ5IHNwZWNpZXMgb2YgYmFjdGVyaWEgaW4gdGhlIHdvcmxkLiAKClRoZSBSRkUgbGlicmFyeSwgc2VlbXMgdG8gdXNlIG1hbnkgb2YgdGhlIHByZS1leGlzdGluZyBmdW5jdGlvbnMgaW4gdGhlIHJhbmRvbUZvcmVzdCBsaWJyYXJ5LCBjb21iaW5lZCB3aXRoIGdncGxvdCwgYW5kIHNvbWUgb3RoZXIgYWRkaXRpb25hbCBsaWJyYXJpZXMgdG8gY3JlYXRlIGJldHRlciBsb29raW5nIGFuZCBlYXN5IHRvIHVuZGVyc3RhbmQgZ3JhcGhzLiAKRXZlbiB0aG91Z2ggdGhlIHJlc3VsdHMgYXJlIG5vdCBhbGwgdGhhdCBkaWZmZXJlbnQgZnJvbSB0aGUgb25lcyB3ZSBnb3QgZnJvbSBSRkUsIHdlIGFsc28gZGVjaWRlZCB0byB1c2UgdGhlIFBEUCBsaWJyYXJ5IGFuZCBzYXZlIGEgY29tcGFyaXNvbiBiZXR3ZWVuIHRoZSBSRkUgYW5kIFBEUCBncmFwaHMgaW4gW3BkcF92c19yZmUucGRmXSguL1Jlc3VsdHMvcGRwX3ZzX3JmZS5wZGYpLgpXZSBhbHNvIHVzZWQgdGhlIFBEUCBsaWJyYXJ5IHRvIGdlbmVyYXRlIHBhcnRpYWwgZGVwZW5kYW5jZSBwbG90IGluZGl2aWR1YWxseSBmb3IgZWFjaCBvZiB0aGUgNyBmZWF0dXJlcywgYnV0IHRoZSByZXN1bHRzIHdlcmUgbm90IGFzIHBvc2l0aXZlIGFzIHdlIGhhZCBob3BlIGluIHRoZSBzZW5zZSB0aGF0IHdlIGRpZG4ndCBnZXQgYW55IHJlYWwgaW5zaWdodHMgaW50byBzUk5BcyBvciBvdXIgbW9kZWwuIApUaGUgZ3JhcGhzIGZvciB0aGVzZSBpbmRpdmlkdWFsIFBEUCBwbG90cyBjYW4gYmUgZm91bmQgaW4gW3BkcF9pbmRpdmlkdWFsX2ZlYXR1cmVzLnBkZl0oLi9SZXN1bHRzL3BkcF92c19yZmUucGRmKS4gCgoKIyBWSSkgTElNRQoKKipMSU1FIEZVTkNUSU9OIEFSR1VNRU5UUyoqCgoqIHgJICAgICAgICAgICAgICBUaGUgdHJhaW5pbmcgZGF0YSB1c2VkIGZvciB0cmFpbmluZyB0aGUgbW9kZWwgdGhhdCBzaG91bGQgYmUgZXhwbGFpbmVkLgoqIG1vZGVsCSAgICAgICAgICBUaGUgbW9kZWwgd2hvc2Ugb3V0cHV0IHNob3VsZCBiZSBleHBsYWluZWQKKiBiaW5fY29udGludW91cyAgU2hvdWxkIGNvbnRpbnVvdXMgdmFyaWFibGVzIGJlIGJpbm5lZCB3aGVuIG1ha2luZyB0aGUgZXhwbGFuYXRpb24KKiBuX2JpbnMJICAgICAgICBUaGUgbnVtYmVyIG9mIGJpbnMgZm9yIGNvbnRpbnVvdXMgdmFyaWFibGVzIGlmIGJpbl9jb250aW51b3VzID0gVFJVRQoqIHF1YW50aWxlX2JpbnMJICBTaG91bGQgdGhlIGJpbnMgYmUgYmFzZWQgb24gbl9iaW5zIHF1YW50aWxlcyBvciBzcHJlYWQgZXZlbmx5IG92ZXIgdGhlIHJhbmdlIG9mCiAgICAgICAgICAgICAgICAgICB0aGUgdHJhaW5pbmcgZGF0YQoqIHVzZV9kZW5zaXR5ICAgICBJZiBiaW5fY29udGludW91cyA9IEZBTFNFIHNob3VsZCBjb250aW51b3VzIGRhdGEgYmUgc2FtcGxlZCB1c2luZyBhIGtlcm5lbCBkZW5zaXR5IAogICAgICAgICAgICAgICAgZXN0aW1hdGlvbi4gSWYgbm90LCBjb250aW51b3VzIGZlYXR1cmVzIGFyZSBleHBlY3RlZCB0byBmb2xsb3cgYSBub3JtYWwgZGlzdHJpYnV0aW9uLgoKKipMSU1FIFNFVFRJTkdTIEpVU1RJRklDQVRJT05TKioKCk5PVEU6IG1hbnkgb2YgdGhlc2Ugc2V0dGluZ3Mgd2VyZSBvYnRhaW5lZCB0aHJvdWdoIHRyaWFsIGFuZCBlcnJvciwgYW5kL29yIGJhc2VkIG9uIHRoZSBzdWdnZXN0aW9ucyBwcm92aWRlZCBieSBMSU1FIGFuZCBpdHMgbGlicmFyaWVzLiBTaW5jZSBMSU1FJ3MgcmVzdWx0cyBhcmUgbm90IGFsd2F5cyB0aGUgc2FtZSwgd2UgdXNlZCB0aGUgc2V0dGluZ3MgdGhhdCAgZ2VuZXJhdGVkIHRoZSBtb3N0IGNvbnNpc3RlbnQgYW5kIHZhbGlkIGV4cGxhbmF0aW9ucyBkdXJpbmcgb3VyIGV4cGVyaW5tZW50cy4gT2YgY291cnNlLCB3ZSBlbmNvdXJhZ2Ugb3RoZXJzIHRvIHRyeSBvdXQgb3RoZXIgc2V0dGluZ3MgYW5kIGV4cGVyaW5tZW50IGZ1cnRoZXIuCiogU2V0dGluZ3MgZm9yICJleHBsYWluZXIiCiAgICAqICBiaW5fY29udGludW91cyA9IFRSVUUgLS0+IE1ha2VzIGV4cGxhbmF0aW9uIGVhc2llciwgYW5kIGhhdmluZyB0aGlzIHNldHRpbmcgYXMgdHJ1ZSBoZWxwZWQgaW4gb2J0YWluaW5nIG1vcmUgY29uc2lzdGVudCBhbmQgcmVsaWFibGUgcmVzdWx0cy4KICAgICogIHF1YW50aWxlX2JpbnMgPSBGQUxTRSAtLT4gc2V0dGluZyB0aGlzIHRvIFRSVUUgZ2VuZXJhdGVzIGFuIGVycm9yIGIvYyAgc2FtZVN0cmFuZCBhbmQgc2FtZURvd25TdHJhbmQgZG9uJ3QgaGF2ZSBlbm91Z2ggdmFyaWFuY2UgZm9yIHRoZSBhbGdvcml0aG0gdG8gdXNlIHF1YW50aWxlIGJpbmluZy4gIAogICAgKiAgdXNlX2RlbnNpdHkgPSBUUlVFIC0tPiB3aXRoIGJpbl9jb250aW51b3VzIHNldCB0byBUUlVFLCB0aGlzIHNldHRpbmcgYmVjb21lcyBpcnJlbGV2YW50LiBJbiBzZWN0aW9uIDMuNCwgd2Ugc2hvd2VkIHRoYXQgdGhlIGZlYXR1cmVzIGRvbid0IGZvbGxvdyBhIG5vcm1hbCBkaXN0cmlidXRpb24uIFdlIHRyaWVkIHNldHRpbmcgYmluX2NvbnRpbnVvdXMgPSBGQUxTRSBhbmQgdXNlX2RlbnNpdHkgPSBUUlVFLCBidXQgdGhpcyBnZW5lcmF0ZWQgY29udHJhZGljdG9yeSBleHBsYW5hdGlvbnMgdGhhdCBkaWRuJ3QgbWFrZSBzZW5zZS4gCiAgICAqICBuX2JpbnMgPSAxMCAtLT4gVGhlIGRlZmF1bHQgdmFsdWUgd2Fzbid0IGdpdmluZyB1cyBjb25zaXN0ZW50IHJlc3VsdHMsIHNvIHdlIGJ1bXBlZCB1cCB0aGUgbnVtYmVyIHRvIDEwIGFuZCBzdGFydGVkIGdldHRpbmcgc2VlbWluZ2x5IGdvb2QgcmVzdWx0cwoKKiBTZXR0aW5ncyBmb3IgImV4cGxhaW4iCiAgICAqICBuX2xhYmVscyA9IDEgICAgICAgICAgLS0+IHdlIG9ubHkgaGF2ZSAxIGxhYmVsIAogICAgKiAgbl9mZWF0dXJlcyA9IDcgICAgICAgIC0tPiB3ZSB3YW50ZWQgb3VyIGV4cGxhbmF0aW9ucyB0byBiZSBiYXNlZCBvbiBhbGwgNyBmZWF0dXJlcyAKICAgICogIG5fcGVybXV0YXRpb25zID0gMjAwMCAtLT4gVGhpcyBudW1iZXIgd2FzIHNldCB0aHJvdWdoIHRyaWFsIGFuZCBlcnJvci4gSGF2aW5nIGEgc21hbGwgbnVtYmVyIG9mIHBlcm11dGF0aW9ucyBkb2VzIG5vdCBnZW5lcmF0ZSBjb25zaXN0ZW4gcmVzdWx0cywgc28gYSBoaWdoIG51bWJlciBvZiBwZXJtdXRhdGlvbnMgaXMgcmVjb21tZW5kZWQuIDIwMDAganVzdCBzZWVtZWQgdG8gaGF2ZSB5aWVsZGVkIGdvb2QgcmVzdWx0cywgc28gd2UgZGVjaWRlZCB0byBrZWVwIHRoaXMgdmFsdWUuIElmIHNwZWVkIGlzIGEgY29uY2VybiwgbG93ZXJpbmcgdGhpcyBudW1iZXIgdG8gMTAwMCBzaG91bGQgc3RpbGwgeWllbGQgZ29vZCByZXN1bHRzLiAKICAgICogIGRpc3RfZnVuID0gImdvd2VyIiAgICAtLT4gTElNRSdzIGRlZmF1bHQgZGlzdF9mdW4sIGFuZCBzaW5jZSB3ZSBoYXZlIGNhdGVnb3JpY2FsIGZlYXR1cmVzIChzYW1lU3RyYW5kIGFuZCBzYW1lRG93blN0cmFuZCksIGFjY29yZGluZyB0byBMSU1FJ3MgZG9jdW1lbnRhdGlvbiwgImdvd2VyIiBpcyB0aGUgZnVuY3Rpb25zIG9mIGNob2ljZSB3aGVuIGRlYWxpbmcgd2l0aCBhIG1peHR1cmUgb2YgbnVtZXJpY2FsIGFuZCBjYXRlZ29yaWNhbCBmZWF0dXJlcwogICAgKiAgU2lkZSBOb3RlOiBXZSB0cmllZCBhIGNvdXBsZSBvZiBydW5zIG9mIHRoZSBzY3JpcHQgd2l0aCBldWNsaWRlYW4gZGlzdGFuY2UsIGJ1dCBpdCBkaWRuJ3Qgc2VlbSB0byBoYXZlIG11Y2ggb2YgYW4gaW1wYWN0LiBBZGRpdGlvbmFsbHksIHRoZSBkaWZmZXJlbnQgdHlwZXMgb2Ygc2V0dGluZ3MsIHN1Y2ggYXMgZGlmZmVyZW50IGdvd2VyIHBvd2Vycywga2VybmVsIHdpZHRocywgYW5kIGRpc3RhbmNlIGZ1bmN0aW9ucywgdGhhdCBjYW4gYmUgdHdlYWtlZCBpbiBMSU1FIG1lYW5zIHRoYXQgYSAxMDAlIHRob3JvdWdoIGFuYWx5c2lzIG9mIGV2ZXJ5IHBvc3NpYmlsaXR5IG1heSBiZSBuZWFybHkgaW1wb3NzaWJsZS4gRm9yIHRoaXMgcmVhc29uLCB3ZSB0cmllZCB0byBzdGljayB0byB0aGUgZGVmYXVsdCBzZXR0aW5ncyBhcyBvZmZlcmVkIGJ5IExJTUUuIAoKCiMjIDE4LiBMSU1FIFBSRS1TRVRVUAojIyMgMTguMSBMSU1FIGNvbXBhdGFiaWxpdHkgZnVuY3Rpb24KClRoZSBvcmlnaW5hbCBtb2RlbCB3YXMgYnVpbHQgdXNpbmcgdGhlIHJhbmRvbUZvcmVzdCBsaWJyYXJ5IGZvdW5kIGluIENSQU4uIEhvd2V2ZXIsIGV2ZW4gdGhvdWdoIExJTUUgaXMgc3VwcG9zZWQgdG8gYmUgbW9kZWwgYWdub3N0aWMsIGl0J3MgY3VycmVudCBSIGltcGxlbWVudGF0aW9uIG9ubHkgc3VwcG9ydHMgY2VydGFpbiBtb2RlbHMgb3V0IG9mIHRoZSBib3guIFRvIGV4dGVuZCBzdXBwb3J0IG9mIHRoZSBMSU1FIGxpYnJhcnkgdG8gb3RoZXIgbW9kZWxzLCB0aGUgZm9sbG93aW5nIGZ1bmN0aW9ucyBuZWVkIHRvIGJlIGRlZmluZWQgZm9yIGVhY2ggIHVuc3VwcG9ydGVkIG1vZGVsIHR5cGUuIAoKYGBge3J9Cm1vZGVsX3R5cGUucmFuZG9tRm9yZXN0IDwtIGZ1bmN0aW9uKHgsLi4uKSAnY2xhc3NpZmljYXRpb24nCgpwcmVkaWN0X21vZGVsLnJhbmRvbUZvcmVzdCA8LSBmdW5jdGlvbih4LCBuZXdkYXRhLCB0eXBlLCAuLi4pIHsKICByZXMgPC0gcHJlZGljdCh4LCBuZXdkYXRhID0gbmV3ZGF0YSwgdHlwZSA9ICJwcm9iIikKICBzd2l0Y2goCiAgICB0eXBlLAogICAgcmF3ID0gZGF0YS5mcmFtZShSZXNwb25zZSA9IGlmZWxzZShhcy52ZWN0b3IocmVzWywyXSkgPiAwLjUsICIxIiwgIjAiKSwgc3RyaW5nc0FzRmFjdG9ycyA9IEZBTFNFKSwKICAgIHByb2IgPSBhcy5kYXRhLmZyYW1lKHJlcywgY2hlY2submFtZXMgPSBGKSAKICApCn0KYGBgCgojIyMgMTguMiBEYXRhIHRvIEV4cGxhaW4KCipzYW1wbGVEYXRhLCBzYW1wbGVEYXRhX3NjYWxlZCwqIGFuZCAqc2FtcGxlRGF0YV9ub3JtKiBhbGwgY29udGFpbiB0aGUgc2FtZSBmZWF0dXJlcywgYW5kIGhlbmNlLCB3ZSBzaG91bGQgYmUgZ2V0dGluZyBzaW1pbGFyIGV4cGxhbmF0aW9ucyBpbiBhbGwgb2Ygb3VyIHJlc3VsdHMuIFdlIGFsc28gYXBwbHkgTElNRSB0byB0aGUgc2FtZSBpbnN0YW5jZSA0IHRpbWVzLCB3aXRoIHRoZSBwdXJwb3NlIG9mIHRlc3RpbmcgdGhlIHZhbGlkaXR5IGFuZCBjb25zaXN0ZW5jeSBvZiBMSU1FJ3MgZXhwbGFuYXRpb25zLiAKCmBgYHtyfQpzYW1wbGVEYXRhIDwtIHNsdDJkYXRhWzEsXSAgIyBBIHRydWUgc2FtcGxlCnNhbXBsZURhdGFbYygyOjQpLF0gPC0gc2x0MmRhdGFbMSxdCnNhbXBsZURhdGFbNSxdIDwtIHNsdDJkYXRhWzE5ODYsXSAjIEEgZmFsc2Ugc2FtcGxlCnNhbXBsZURhdGFbYyg2OjgpLF0gPC0gc2x0MmRhdGFbMTk4NixdCnNhbXBsZURhdGEKCnNhbXBsZURhdGFfc2NhbGVkIDwtIHNsdDJkYXRhX3NjYWxlZFsxLF0gIApzYW1wbGVEYXRhX3NjYWxlZFtjKDI6NCksXSA8LSBzbHQyZGF0YV9zY2FsZWRbMSxdCnNhbXBsZURhdGFfc2NhbGVkWzUsXSA8LSBzbHQyZGF0YV9zY2FsZWRbMTk4NixdIApzYW1wbGVEYXRhX3NjYWxlZFtjKDY6OCksXSA8LSBzbHQyZGF0YV9zY2FsZWRbMTk4NixdCnNhbXBsZURhdGFfc2NhbGVkCgpzYW1wbGVEYXRhX25vcm0gPC0gc2x0MmRhdGFfbm9ybVsxLF0gIApzYW1wbGVEYXRhX25vcm1bYygyOjQpLF0gPC0gc2x0MmRhdGFfbm9ybVsxLF0Kc2FtcGxlRGF0YV9ub3JtWzUsXSA8LSBzbHQyZGF0YV9ub3JtWzE5ODYsXSAKc2FtcGxlRGF0YV9ub3JtW2MoNjo4KSxdIDwtIHNsdDJkYXRhX25vcm1bMTk4NixdCnNhbXBsZURhdGFfbm9ybQpgYGAKCiMjIDE5LiBMSU1FIHdpdGggUmFuZG9tRm9yZXN0IExpYnJhcnkKIyMjIDE5LjEgSW4gb3JpZ2luYWwgTW9kZWwgYXMgaXMKCldoaWxlIG9uIG9jY2Fzc2lvbiBhbGwgcmVzdWx0cyBtYXkgbG9vayBzaW1pbGFyLCB3ZSByZWNvbW1lbmQgcnVubmluZyB0aGlzIHBhcnRpY3VsYXIgc2VjdGlvbiBhIGZldyB0aW1lcywgYW5kIHlvdSB3aWxsIG5vdGljZSBzaWduaWZpY2FudCBjaGFuZ2VzLCBsaWtlIGZlYXR1cmVzIHRoYXQgbWF5IHN1cHBvcnQgdGhlIGV4cGxhbmF0aW9uIGluIG9uZSBwbG90LCBidXQgY29udHJhZGljdCBpdCBpbiB0aGUgbmV4dC4gCmBgYHtyfQpwcmVkaWN0KG9yaWdSRiwgc2FtcGxlRGF0YVtjKDEsNSksLWMoODo5KV0sIHR5cGUgPSAicHJvYiIpCmxpbWVfZXhwbGFpbmVyX29yaWcgPC0gbGltZShhcy5kYXRhLmZyYW1lKHRyYWluRGF0YVssYygxOjcpXSksIG9yaWdSRiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJpbl9jb250aW51b3VzID0gVFJVRSwgcXVhbnRpbGVfYmlucz1GQUxTRSwgdXNlX2RlbnNpdHkgPSBUUlVFKQpsaW1lX2V4cGxhbmF0aW9uc19vcmlnIDwtIGV4cGxhaW4oYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhW2MoMTo0KSxjKDE6NyldKSwgbGltZV9leHBsYWluZXJfb3JpZywgbl9sYWJlbHMgPSAxLCBuX2ZlYXR1cmVzID0gNywgbl9wZXJtdXRhdGlvbnMgPSAyMDAwKQpwbG90X2ZlYXR1cmVzKGxpbWVfZXhwbGFuYXRpb25zX29yaWcpCmxpbWVfZXhwbGFuYXRpb25zX29yaWcgPC0gZXhwbGFpbihhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFbYyg1OjgpLGMoMTo3KV0pLCBsaW1lX2V4cGxhaW5lcl9vcmlnLCBuX2xhYmVscyA9IDEsIG5fZmVhdHVyZXMgPSA3LCBuX3Blcm11dGF0aW9ucyA9IDIwMDApCnBsb3RfZmVhdHVyZXMobGltZV9leHBsYW5hdGlvbnNfb3JpZykKCmBgYAoKIyMjIDE5LjIgSW4gb3JpZ2luYWwgc2NhbGVkIE1vZGVsCgpgYGB7cn0KcHJlZGljdChvcmlnUkZfc2NhbGVkLCBzYW1wbGVEYXRhX3NjYWxlZFtjKDEsNSksLWMoODo5KV0sIHR5cGUgPSAicHJvYiIpCgpsaW1lX2V4cGxhaW5lcl9vcmlnX3NjYWxlZCA8LSBsaW1lKGFzLmRhdGEuZnJhbWUodHJhaW5EYXRhX3NjYWxlZFssIGMoMTo3KV0pLCBvcmlnUkZfc2NhbGVkLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJpbl9jb250aW51b3VzID0gVFJVRSwgcXVhbnRpbGVfYmlucz1GQUxTRSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdXNlX2RlbnNpdHkgPSBGQUxTRSwgbl9iaW5zID0gMTApCm5vcm1fZGlzdCA8LSBhcy5kYXRhLmZyYW1lKGxpbWVfZXhwbGFpbmVyX29yaWdfc2NhbGVkJGJpbl9jdXRzKQoKbGltZV9leHBsYWluZXJfb3JpZ19zY2FsZWQgPC0gbGltZShhcy5kYXRhLmZyYW1lKHRyYWluRGF0YV9zY2FsZWRbLCBjKDE6NyldKSwgb3JpZ1JGX3NjYWxlZCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiaW5fY29udGludW91cyA9IFRSVUUsIHF1YW50aWxlX2JpbnM9RkFMU0UsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVzZV9kZW5zaXR5ID0gVFJVRSwgbl9iaW5zID0gMTApCm5vdF9ub3JtX2Rpc3QgPC0gYXMuZGF0YS5mcmFtZShsaW1lX2V4cGxhaW5lcl9vcmlnX3NjYWxlZCRiaW5fY3V0cykKCiMgVGhlIGNvcnJlbGF0aW9ucyBoZXJlIHNob3cgdGhhdCB0aGUgYmlucyB1c2VkIGJ5IHRoZSBleHBsYWluZXIgYXJlIHRoZSBzYW1lIHJlZ2FyZGxlc3Mgb2Ygd2hhdCB2YWx1ZSBpcyBnaXZlbiB0byB0aGUgdXNlX2RlbnNpdHkgcGFyYW1ldGVyLgpjb3Iobm9ybV9kaXN0WywiU1MiXSxub3Rfbm9ybV9kaXN0WywiU1MiXSkKY29yKG5vcm1fZGlzdFssIlBvczEwd3J0c1JOQVN0YXJ0Il0sbm90X25vcm1fZGlzdFssIlBvczEwd3J0c1JOQVN0YXJ0Il0pCmNvcihub3JtX2Rpc3RbLCJEaXN0VGVybSJdLG5vdF9ub3JtX2Rpc3RbLCJEaXN0VGVybSJdKQpjb3Iobm9ybV9kaXN0WywiRGlzdGFuY2UiXSxub3Rfbm9ybV9kaXN0WywiRGlzdGFuY2UiXSkKY29yKG5vcm1fZGlzdFssInNhbWVTdHJhbmQiXSxub3Rfbm9ybV9kaXN0Wywic2FtZVN0cmFuZCJdKQpjb3Iobm9ybV9kaXN0WywiRG93bkRpc3RhbmNlIl0sbm90X25vcm1fZGlzdFssIkRvd25EaXN0YW5jZSJdKQpjb3Iobm9ybV9kaXN0Wywic2FtZURvd25TdHJhbmQiXSxub3Rfbm9ybV9kaXN0Wywic2FtZURvd25TdHJhbmQiXSkKCmxpbWVfZXhwbGFpbmVyX29yaWdfc2NhbGVkJGJpbl9jdXRzICMgQWxsIHRoZSBmZWF0dXJlcyBzZWVtIHRvIGJlIGV2ZW5seSBzcGxpdCBpbnRvIDEwLCBpZSwgYSBkZW5zaXR5IGlzIG5vdCB1c2VkIGZvciB0aGUgYmluIHNwbGl0dGluZy4gIAoKbGltZV9leHBsYW5hdGlvbnNfb3JpZ19zY2FsZWQgPC0gZXhwbGFpbihhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFfc2NhbGVkW2MoMTo0KSxjKDE6NyldKSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbGltZV9leHBsYWluZXJfb3JpZ19zY2FsZWQsIG5fbGFiZWxzID0gMSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9mZWF0dXJlcyA9IDcsIG5fcGVybXV0YXRpb25zID0gMjAwMAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19vcmlnX3NjYWxlZCkKCmxpbWVfZXhwbGFuYXRpb25zX29yaWdfc2NhbGVkIDwtIGV4cGxhaW4oYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhX3NjYWxlZFtjKDU6OCksYygxOjcpXSksIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWVfZXhwbGFpbmVyX29yaWdfc2NhbGVkLCBuX2xhYmVscyA9IDEsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5fZmVhdHVyZXMgPSA3LCBuX3Blcm11dGF0aW9ucyA9IDIwMDAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICApCnBsb3RfZmVhdHVyZXMobGltZV9leHBsYW5hdGlvbnNfb3JpZ19zY2FsZWQpCgpgYGAKICAKIyMjIDE5LjMgSW4gb3JpZ2luYWwgbm9ybWFsaXplZCBNb2RlbAoKYGBge3J9CnByZWRpY3Qob3JpZ1JGX25vcm0sIHNhbXBsZURhdGFfbm9ybVtjKDEsNSksLWMoODo5KV0sIHR5cGUgPSAicHJvYiIpCmxpbWVfZXhwbGFpbmVyX29yaWdfbm9ybSA8LSBsaW1lKGFzLmRhdGEuZnJhbWUodHJhaW5EYXRhX25vcm1bLCBjKDE6NyldKSwgb3JpZ1JGX25vcm0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmluX2NvbnRpbnVvdXMgPSBUUlVFLCBxdWFudGlsZV9iaW5zPUZBTFNFLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1c2VfZGVuc2l0eSA9IFRSVUUsIG5fYmlucyA9IDEwKQoKbGltZV9leHBsYWluZXJfb3JpZ19ub3JtJGJpbl9jdXRzICMgYWxsIGJpbnMgYXJlIHNwbGl0IGV2ZW5seSwgZnJvbSAwIHRvIDEsIGluIGluY3JlbWVudHMgb2YgMC4xLCBhcyB3b3VsZCBiZSBleHBlY3RlZAoKbGltZV9leHBsYW5hdGlvbnNfb3JpZ19ub3JtIDwtIGV4cGxhaW4oYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhX25vcm1bYygxOjQpLGMoMTo3KV0pLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsaW1lX2V4cGxhaW5lcl9vcmlnX25vcm0sIG5fbGFiZWxzID0gMSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9mZWF0dXJlcyA9IDcsIG5fcGVybXV0YXRpb25zID0gMjAwMAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICkKCnBsb3RfZmVhdHVyZXMobGltZV9leHBsYW5hdGlvbnNfb3JpZ19ub3JtKQoKbGltZV9leHBsYW5hdGlvbnNfb3JpZ19ub3JtIDwtIGV4cGxhaW4oYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhX25vcm1bYyg1OjgpLGMoMTo3KV0pLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsaW1lX2V4cGxhaW5lcl9vcmlnX25vcm0sIG5fbGFiZWxzID0gMSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9mZWF0dXJlcyA9IDcsIG5fcGVybXV0YXRpb25zID0gMjAwMAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19vcmlnX25vcm0pCgoKCmBgYAoKIyMgMjAuIExJTUUgaW4gT3JpZyBSRiBDb25jbHVzaW9ucwoKV2UgdHJpZWQgYXBwbHlpbmcgTElNRSB0byB0aGUgT3JpZyBSRihpbiBwYXJ0IDE1KSBob3BpbmcgaXQgd291bGQgd29yayBsaWtlIGEgcGx1Zy1hbmQtcGxheSBhcHBsaWNhdGlvbi4gClRvIG91ciBzdXJwcmlzZSwgeW91IGNhbid0IGdldCBMSU1FIHRvIHByb3Blcmx5IHdvcmsgd2l0aCB0aGUgIk9yaWdpbmFsIiBSRiBtb2RlbCBiZWNhdXNlIHlvdSBtdXN0IG5vcm1hbGl6ZSwgb3Igc2NhbGUgdGhlIGRhdGEuIApMSU1FIGdlbmVyYXRlcyByYW5kb20gcGVybXV0YXRpb25zIHRvIHRoZSBmZWF0dXJlcyBpbiBvcmRlciB0byBjcmVhdGUgYSBsaW5lYXIgbW9kZWwsIGJ1dCBpZiB0aGUgZmVhdHVyZXMgaGF2ZSBkaWZmZXJlbnQgc2NhbGVzIGFuZCB2YWx1ZXMsIGl0IGlzIHBsYXVzaWJsZSB0aGF0IGZlYXR1cmVzIHdpdGggaGlnaGVyIHNjYWxlcyB3aWxsIHNpbXBseSBvdXR3ZWlnaHQgdGhlIG90aGVyIGZlYXR1cmVzLCBldmVuIGlmIHRoZXkncmUgc2lnbmlmaWNhbnRseSBsZXNzIGltcG9ydGFudCwgYW5kIGhlbmNlIGNyZWF0ZSBza2V3ZWQgbW9kZWxzIGFuZCBleHBsYW5hdGlvbnMuClRvIHRlc3QgdGhpcyBoeXBvdGhlc2lzLCB3ZSBjcmVhdGVkIGEgR0xNIG1vZGVsIGFuZCByZXZpZXdlZCBpdHMgUl4yIHZhbHVlLCB3aGljaCB3YXMgc3VwcmlzaW5nbHkgbG93LCBhbmQgaGVuY2UsIGZpbmlzaGVkIGNvbnZpbmNpbmcgdXMgdGhhdCB0aGUgbGluZWFyIG1vZGVscyBnZW5lcmF0ZWQgYnkgTElNRSB3aXRoIHRoZSBPcmlnUkYgd2VyZSBpbmFkZXF1YXRlLiAKQWRkaXRpb25hbGx5LCBzaW5jZSBMSU1FIHJhbmRvbWl6ZXMgdGhlIHBlcm11dGF0aW9uIHBvaW50cywgYmFzZWQgb24gb3VyIGV4cGVyaW5tZW50cywgaXQgc2VlbXMgdGhhdCBMSU1FIGlzIHVzaW5nIGl0cyBvd24gcmFuZG9taXphdGlvbiBhbGdvcml0aCAoaWUuIHdlIGNvdWxkbid0IHNldCBhIHNlZWQgZm9yIGl0KSwgYW5kIGhlbmNlLCB0aGUgZXhhY3QgcmVzdWx0cyB3ZSBvYnRhaW5lZCB3aXRoIHRoZSBPcmlnUkYgbWF5IG5vdCBiZSBjb21wbGV0ZWx5IHJlcGVhdGFibGUuIEhvd2V2ZXIsIGZvciB0aGUgcGFydGljdWxhciBjYXNlIGluIHBhcnQgMTUsIHRoYXQgbWF5IGJlIGlycmVsZXZhbnQgYXMgdGhlIHBvaW50IHdlIGFyZSB0cnlpbmcgdG8gcHJvdmUgdGhlcmUgaXMgdGhhdCBMSU1FJ3MgZXhwbGFuYXRpb25zIHdoZW4gdGhlIGRhdGEgaXMgbm90IHNjYWxlZCBvciBub3JtYWxpemVkIGFyZSBpbmNvbnNpc3RlbnQsIGFuZCBoZW5jZSBjYW5ub3QgYmUgdHJ1c3RlZC4gIApGb3IgZWFjaCBMSU1FIGV4cGxhbmF0aW9uIHBsb3QsIHNpbmNlIHdlJ3JlIGhhdmluZyBMSU1FIGV4cGxhaW4gdGhlIHNhbWUgaW5zdGFuY2UsIGFsbCA0IHBsb3RzIHNob3VsZCBiZSBpZGVudGljYWwgdG8gZWFjaCBvdGhlci4gCkhvd2V2ZXIsIHdpdGggdGhlIE9yaWdSRiBtb2RlbCwgdGhlcmUncyB1c3VhbGx5IGF0IGxlYXN0IDEgcGxvdCB0aGF0IHJhbmtzIHRoZSBmZWF0dXJlIGNvbnRyaWJ1dGlvbnMgZGlmZmVyZW50bHksIG9yIHRoYXQgbWF5IGhhdmUgYSBmZWF0dXJlICJTdXBwb3J0aW5nIiBpbiBvbmUgcGxvdCwgYnV0ICJDb250cmFkaWN0aW5nIiBpbiB0aGUgbmV4dC4gCkR1ZSB0byBMSU1FJ3MgbmF0dXJlLCBvYnRhaW5pbmcgc2xpZ2h0bHkgZGlmZmVyZW50IHJlc3VsdHMgd2FzIGV4cGVjdGVkLCBidXQgbm90IHRvIGEgZGVncmVlIHRoYXQgd291bGQgbWFrZSB0aGUgZXhwbGFuYXRpb25zIGludmFsaWQuICAKCkJ5IHNjYWxpbmcsIG9yIG5vcm1hbGl6aW5nLCB0aGUgZGF0YSwgd2Ugd2VyZSBhYmxlIHRvIG9idGFpbiBtb3JlIGNvbnNpc3RlbnQgcmVzdWx0cyAoaWUuIGZlYXR1cmVzIHRoYXQgc3VwcG9ydCBvciBjb250cmFkaWN0IGEgcHJlZGljdGlvbiwgYmVoYXZlIHRoZSBzYW1lIGluIGFsbCB0aGUgY29ycmVzcG9uZGluZyA0IHBsb3RzLCBhbHRob3VnaCB0aGVpciBpbXBvcnRhbmNlIHJhbmtpbmcgbWF5IGJlIGRpZmZlcmVudCksIGFuZCBpbiBtb3N0IGNhc2VzLCB0aGUgcmFua2luZ3Mgb2YgZmVhdHVyZSBjb250cmlidXRpb25zIGFzIHBsb3R0ZWQgYnkgTElNRSBhY3R1YWxseSByZW1haW5lZCB0aGUgc2FtZS4gCkxJTUUgYmVoYXZlcyBwcm9wZXJseSBpZiB0aGUgZGF0YSBpcyBzY2FsZWQgYW5kIG5vcm1hbGl6ZWQuCgpBbiBhZGRpdGlvbmFsIGludGVyZXN0aW5nIHRha2UtYXdheSwgaXMgdGhhdCBzYW1lU3RyYW5kIGFuZCBzYW1lRG93blN0cmFuZCBhcmUgY29uc2lkZXJlZCB0aGUgbGVhc3QgaW1wb3J0YW50IGZlYXR1cmVzLCBhbmQgc2VlbSB0byBiZWNvbWUgY29tcGxldGVseSBpcnJlbGV2YW50IGZvciBwcmVkaWN0aW9ucyB3aGVyZSB0aGUgaW5zdGFuY2UgaXMgRkFMU0UgYXMgTElNRSBzaW1wbHkgZXhjbHVkZSB0aGlzIGZlYXR1cmUgLiAKCgojIyAyMS4gTElNRSBmb3IgdGhlIEgyTyBSRiBtb2RlbHMKCkluaXRpYWwgc3RydWdnbGVzIHdpdGggZ2V0dGluZyBMSU1FIHNldCB1cCB3aXRoIHRoZSBvcmlnaW5hbCByYW5kb20gZm9yZXN0IGxlYWQgdXMgdG8gYmVsaWV2ZSB0aGF0IHNpbmNlIFJhbmRvbUZvcmVzdCB3YXNuJ3QgYSBzdXBwb3J0ZWQgbWV0aG9kIGJ5IGRlZmF1bHQgaW4gdGhlIExpbWUgTGlicmFyeSwgdGhhdCBtYXliZSB1c2luZyBhIHN1cHBvcnRlZCBtZXRob2Qgd291bGQgaGVscCB1cyBvdmVyY29tZSB0aGVzZSBwcm9ibGVtcy4gCkR1cmluZyBvdXIgaW52ZXN0aWdhdGlvbiwgd2UgZGlzY292ZXJlZCB0aGF0IEgyTyBvZmZlcmVkIGFkZGl0aW9uYWwgSU1zIChQRFBzIGFuZCBTSEFQKSwgYW5kIHRoYXQgdGhlaXIgYWxnb3JpdGhtcyB3ZXJlIG5hdGl2ZWx5IHN1cHBvcnRlZCBieSB0aGUgTGltZSBsaWJyYXJ5LiAKVGhpbmtpbmcgd2UgY291bGQga2lsbCAzIGJpcmRzICh0ZXN0IDMgSU1zKSB3aXRoIG9uZSBzdG9uZSAob25lIHBhY2thZ2UpLCB3ZSBwcm9jZWVkZWQgdG8gY3JlYXRlIHRoZSBlcXVpdmFsZW50IHByb3h5IG1vZGVscyBpbiBIMk8gYW5kIHJ1biB0aGUgY29ycmVzcG9uZGluZyB0ZXN0LiAKVGhlIGZpcnN0IEgyTyBSRiBtb2RlbCB3YXMgY3JlYXRlZCBwcmlvciB0byByZWFsaXppbmcgd2UgbmVlZGVkIHRvIGhhdmUgbm9ybWFsaXplZCBhbmQgc2NhbGVkIGRhdGEgc2V0cywgYXMgdGhlc2UgZGF0YSBwcmVwcm9jZXNzaW5nIHRlY2huaXF1ZXMgYXJlIG5vcm1hbGx5IG5vdCBuZWNlc3NhcnkgZm9yIHJhbmRvbSBmb3Jlc3RzLiAKRnVydGhlciBpbnZlc3RpZ2F0aW9ucyBhbmQgdGVzdGluZyBhbGxvd2VkIHVzIHRvIGRpc2NvdmVyIHRoYXQgbm9ybWFsaXppbmcgYW5kIHNjYWxpbmcgdGhlIGRhdGEgYWxsb3dlZCBMSU1FIHRvIGdlbmVyYXRlIHJlYXNvbmFibGUgZXhwbGFuYXRpb25zIHdpdGggdGhlIE9yaWdSRiBtb2RlbC4KRXZlbnR1YWxseSB3ZSBkZWNpZGVkIHRvIGNyZWF0ZSA1IG1vZGVscyBpbiB0b3RhbCwgMiB3aXRoIHRoZSBvcmlnaW5hbCBSYW5kb21Gb3Jlc3QgbGlicmFyeSwgYW5kIDMgd2l0aCB0aGUgSDJPIGxpYnJhcnkuIApFdmVuIHRob3VnaCB0aGUgcGVyZm9ybWFuY2UgbWV0cmljcyBmb3IgdGhlIHNjYWxlZCBhbmQgbm9ybWFsaXplZCBIMk8gUkYgbW9kZWxzIGRpZCBub3QgbWF0Y2ggdGhvc2Ugb2YgdGhlIE9yaWdSRiwgd2UgZGVjaWRlZCB0byBzdGlsbCBpbmNsdWRlIHRoZW0gaW4gdGhlIGZvbGxvd2luZyBzZWN0aW9ucy4gCgojIyMgMjEuMSBIMk8gUkYgTW9kZWwKCmBgYHtyfQpoMm8ucHJlZGljdChvYmplY3QgPSByZmgybywgbmV3ZGF0YSA9IGFzLmgybyhzYW1wbGVEYXRhW2MoMSw4KSxjKDE6NyldKSApCmxpbWVfZXhwbGFpbmVyX3JmaDJvIDwtIGxpbWUoIGFzLmRhdGEuZnJhbWUoIHRyYWluRGF0YVssYygxOjcpXSApLCByZmgybywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmluX2NvbnRpbnVvdXMgPSBUUlVFLCBxdWFudGlsZV9iaW5zID0gRkFMU0UsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5fYmlucyA9IDEwCikKbGltZV9leHBsYW5hdGlvbnNfcmZoMm8gPC0gZXhwbGFpbiggYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhW2MoMTo0KSxjKDE6NyldICksbGltZV9leHBsYWluZXJfcmZoMm8sIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2xhYmVscyA9IDEsICBuX2ZlYXR1cmVzID0gNywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5fcGVybXV0YXRpb25zID0gMjAwMCApCnBsb3RfZmVhdHVyZXMobGltZV9leHBsYW5hdGlvbnNfcmZoMm8pIAoKbGltZV9leHBsYW5hdGlvbnNfcmZoMm8gPC0gZXhwbGFpbiggYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhW2MoNTo4KSxjKDE6NyldICksbGltZV9leHBsYWluZXJfcmZoMm8sIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2xhYmVscyA9IDEsICBuX2ZlYXR1cmVzID0gNywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5fcGVybXV0YXRpb25zID0gMjAwMCApCnBsb3RfZmVhdHVyZXMobGltZV9leHBsYW5hdGlvbnNfcmZoMm8pIAoKYGBgCgojIyMgMjEuMiBJbiBIMk8gUkYgc2NhbGVkIE1vZGVsCgpgYGB7cn0KaDJvLnByZWRpY3Qob2JqZWN0ID0gcmZoMm9fc2NhbGVkLCBuZXdkYXRhID0gYXMuaDJvKHNhbXBsZURhdGFfc2NhbGVkW2MoMSw4KSxjKDE6NyldKSkKbGltZV9leHBsYWluZXJfcmZoMm9fc2NhbGVkIDwtIGxpbWUoIGFzLmRhdGEuZnJhbWUoIHRyYWluRGF0YV9zY2FsZWRbLGMoMTo3KV0gKSwgcmZoMm9fc2NhbGVkLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiaW5fY29udGludW91cyA9IFRSVUUsIHF1YW50aWxlX2JpbnMgPSBGQUxTRSkgCgpsaW1lX2V4cGxhbmF0aW9uc19yZmgyb19zY2FsZWQgPC0gZXhwbGFpbiggYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhX3NjYWxlZFtjKDE6NCksYygxOjcpXSksICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbGltZV9leHBsYWluZXJfcmZoMm9fc2NhbGVkLCBuX2xhYmVscyA9IDEsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2ZlYXR1cmVzID0gNywgbl9wZXJtdXRhdGlvbnMgPSAyMDAwLCBkaXN0X2Z1biA9ICJnb3dlciIgKQpwbG90X2ZlYXR1cmVzKGxpbWVfZXhwbGFuYXRpb25zX3JmaDJvX3NjYWxlZCkKCmxpbWVfZXhwbGFuYXRpb25zX3JmaDJvX3NjYWxlZCA8LSBleHBsYWluKCBhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFfc2NhbGVkW2MoNTo4KSxjKDE6NyldKSwgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsaW1lX2V4cGxhaW5lcl9yZmgyb19zY2FsZWQsIG5fbGFiZWxzID0gMSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5fZmVhdHVyZXMgPSA3LCBuX3Blcm11dGF0aW9ucyA9IDIwMDAsIGRpc3RfZnVuID0gImdvd2VyIiApCnBsb3RfZmVhdHVyZXMobGltZV9leHBsYW5hdGlvbnNfcmZoMm9fc2NhbGVkKQoKYGBgCgojIyMgMjEuMyBMSU1FIHRvIEgyTyBSRiBub3JtYWxpemVkIE1vZGVsIAoKYGBge3J9Cmgyby5wcmVkaWN0KG9iamVjdCA9IHJmaDJvX25vcm0sIG5ld2RhdGEgPSBhcy5oMm8oc2FtcGxlRGF0YV9ub3JtW2MoMSw4KSxjKDE6NyldKSApCmxpbWVfZXhwbGFpbmVyX3JmaDJvX25vcm0gPC0gbGltZSggYXMuZGF0YS5mcmFtZSggdHJhaW5EYXRhX25vcm1bLGMoMTo3KV0gKSwgcmZoMm9fbm9ybSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiaW5fY29udGludW91cyA9IFRSVUUsIHF1YW50aWxlX2JpbnMgPSBGQUxTRSwgdXNlX2RlbnNpdHkgPSBUUlVFKQoKbGltZV9leHBsYW5hdGlvbnNfcmZoMm9fbm9ybSA8LSBleHBsYWluKCBhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFfbm9ybVtjKDE6NCksYygxOjcpXSApLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWVfZXhwbGFpbmVyX3JmaDJvX25vcm0sIG5fbGFiZWxzID0gMSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9mZWF0dXJlcyA9IDcsIG5fcGVybXV0YXRpb25zID0gMjAwMCwgZGlzdF9mdW4gPSAiZ293ZXIiICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19yZmgyb19ub3JtKQoKbGltZV9leHBsYW5hdGlvbnNfcmZoMm9fbm9ybSA8LSBleHBsYWluKCBhcy5kYXRhLmZyYW1lKHNhbXBsZURhdGFfbm9ybVtjKDU6OCksYygxOjcpXSApLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWVfZXhwbGFpbmVyX3JmaDJvX25vcm0sIG5fbGFiZWxzID0gMSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9mZWF0dXJlcyA9IDcsIG5fcGVybXV0YXRpb25zID0gMjAwMCwgZGlzdF9mdW4gPSAiZ293ZXIiICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19yZmgyb19ub3JtKQoKYGBgCgojIyMgMjEuNCBBcHBseWluZyBMSU1FIHRvIEdMTSBNb2RlbCAoQk9OVVMpIAoKYGBge3J9CmxpbWVfZXhwbGFpbmVyX2dsbWgybyA8LSBsaW1lKCBhcy5kYXRhLmZyYW1lKCB0cmFpbkRhdGFbLGMoMTo3KV0gKSxnbG1oMm8sIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmluX2NvbnRpbnVvdXMgPSBUUlVFLCBxdWFudGlsZV9iaW5zID0gRkFMU0UsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdXNlX2RlbnNpdHkgPSBGQUxTRSwgbl9iaW5zID0gMTAgKQpsaW1lX2V4cGxhbmF0aW9uc19nbG1oMm8gPC0gZXhwbGFpbiggYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhW2MoMTo0KSxjKDE6NyldICksIGxpbWVfZXhwbGFpbmVyX2dsbWgybywgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2xhYmVscyA9IDEsICBuX2ZlYXR1cmVzID0gNywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9wZXJtdXRhdGlvbnMgPSAyMDAwLCBkaXN0X2Z1biA9ICJnb3dlciIgICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19nbG1oMm8pCgpsaW1lX2V4cGxhbmF0aW9uc19nbG1oMm8gPC0gZXhwbGFpbiggYXMuZGF0YS5mcmFtZShzYW1wbGVEYXRhW2MoNTo4KSxjKDE6NyldICksIGxpbWVfZXhwbGFpbmVyX2dsbWgybywgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuX2xhYmVscyA9IDEsICBuX2ZlYXR1cmVzID0gNywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbl9wZXJtdXRhdGlvbnMgPSAyMDAwLCBkaXN0X2Z1biA9ICJnb3dlciIgICkKcGxvdF9mZWF0dXJlcyhsaW1lX2V4cGxhbmF0aW9uc19nbG1oMm8pCgpgYGAKCgojIyAyNS4gTElNRSBDT05DTFVTSU9OUyAKCkEga25vd24gcHJvYmxlbSB3aXRoIExJTUUgaXMgdGhhdCBleHBsYW5hdGlvbnMgbWF5IGJlIGluY29uc2lzdGVudCBmcm9tIG9uZSBpbnN0YW5jZSB0byB0aGUgbmV4dCwgZXZlbiB3aGVuIHRoZSBpbnN0YW5jZXMgYXJlIHZlcnkgc2ltaWxhciB0byBlYWNoLiAKV2l0aCB0aGlzIGluIG1pbmQsIGVhY2ggaW5zdGFuY2Ugd2UgdGVzdGVkIHdhcyB0ZXN0ZWQgYXQgbGVhc3QgNCB0aW1lcyAod2UgbWFudWFsbHkgcmFuIHRoaXMgc2VjdGlvbiBvdmVyIGFuZCBvdmVyIGFnYWluKSAsIGFuZCB3aXRoIGEgaGlnaCBlbm91Z2ggbnVtYmVyIG9mIHBlcm11dGF0aW9ucywgd2UgbWFuYWdlZCB0byBnZXQgc29tZXdoYXQgY29uc2lzdGVudCByZXN1bHRzLiAKSG93ZXZlciwgc29tZSBvZiB0aGUgcmVzdWx0cyBvYnRhaW5lZCBmcm9tICBMSU1FIHdlcmUgaGVhdmlseSBmbGF3ZWQgYXMgc29tZXRpbWVzIGl0IHdvdWxkIHlpZWxkIGV4cGxhbmF0aW9ucyB0aGF0IGNvbnRyYWRpY3RlZCB0aGUgcmVzdWx0LiAKRm9yIHNvbWUgcmVhc29uLCB3aGVuIGV4cGxhaW5pbmcgZmFsc2UgaW5zdGFuY2VzLCBMSU1FIHdvdWxkIHRlbmQgdG8gbGVhdmUgb3V0IHNhbWVTdHJhbmQgYW5kIHNhbWVEb3duU3RyYW5kIGZyb20gdGhlIGV4cGxhbmF0aW9uLgpBZGRpdGlvbmFsbHksIGEgR0xNIHdhcyBhbHNvIHRyYWluZWQgdG8gYXR0ZW1wdCB0byBtZWFzdXJlIHRoZSB2YXJpYW5jZSBvZiB0aGUgZGF0YSwgYnV0IHRoZSByZXN1bHRzIGZyb20gdGhlIEdMTSB5aWVsZGVkIGFuIFJeMiB2YWx1ZSBiZXR3ZWVuIDAuMiBhbmQgMC4zLCB3aXRoIGEgbWF4IHJlY2FsbCBvZiAwLjA3LgpJbiBvdGhlciB3b3JkcywgdGhlIEdMTSBtb2RlbCB3YXMgb25seSBwcmVkaWN0aW5nIGZhbHNlIGluc3RhbmNlcywgYW5kIHNpbmNlIExJTUUgdXNlcyBsaW5lYXIgbW9kZWxzIGZvciBpdHMgZXhwbGFuYXRpb25zLCBiYXNlZCBvbiBvdXIgb2JzZXJ2YXRpb25zLCBpdCBpcyBzYWZlIHRvIGNvbmNsdWRlIHRoYXQgTElNRSB3YXMgZG9pbmcgdGhlIHNhbWUuCldoZW5ldmVyIGEgVFJVRSBpbnN0YW5jZSB3YXMgYW5hbHl6ZSwgTElNRSdzIGV4cGxhbmF0aW9ucyB3b3VsZCBzYXkgdGhhdCBtb3N0IGZhY3RvcnMgY29udHJhZGljdCB0aGUgcHJlZGljdGlvbiwgd2hpY2ggaXMgb2J2aW91c2x5IGNvdW50ZXIgaW50dWl0aXZlIGFuZCB1c2VsZXNzLgoKRHVyaW5nIHRoZSBtYWtpbmcgb2YgdGhpcyBwcm9qZWN0LCBzZXZlcmFsIHByb2JsZW1zIHdlcmUgZW5jb3VudGVyZWQgd2hpbGUgdHJ5aW5nIHRvIGFwcGx5IExJTUUgdG8gdGhlIHNSTkEgUkYgbW9kZWwuIApMSU1FIGF0dGVtcHRzIHRvIGV4cGxhaW4gYSBjb21wbGV4IG1vZGVsJ3MgYmVoYXZpb3IgaW4gYSBzbWFsbCByZWdpb24gY29ycmVzcG9uZGluZyB0byBhIHBhcnRpY3VsYXIgaW5zdGFuY2Ugb2YgaW50ZXJlc3QgYnkgYXBwbHlpbmcgYSBsaW5lYXIgbW9kZWwuIApIb3dldmVyLCBpbiB0aGlzIHBhcnRpY3VsYXIgdXNlIGNhc2UsIHRoZSBtaXh0dXJlIG9mIGNhdGVnb3JpY2FsIGFuZCBudW1lcmljYWwgdmFsdWVzLCBwbHVzIHRoZSBkaWZmZXJlbmNlcyBpbiByYW5nZXMgd2l0aGluIGVhY2ggZmVhdHVyZSByZXN1bHRlZCBpbiB2YXJpb3VzIGV4cGVyaW5tZW50cyB3aXRob3V0IGNvbnNpc3RlbnQgcmVzdWx0cy4gCkZvciB0aGlzIHBhcnRpY3VsYXIgdXNlIGNhc2UsIExJTUUgd2FzIGltcGxlbWVudCBpbiBtb2RlbHMgdHJhaW5lZCB3aXRoIHRoZSBvcmlnaW5hbCB0cmFpbmluZyBkYXRhLCBzY2FsZWQgdHJhaW5pbmcgZGF0YSwgYW5kIG5vcm1hbGl6ZWQgdHJhaW5pbmcgZGF0YS4gClNldmVyYWwgYXR0ZW1wdHMgd2VyZSBhbHNvIG1hZGUgYXQgbW9kaWZ5aW5nIHRoZSBhcmd1bWVudHMgZm9yIHRoZSBsaW1lKCkgZnVuY3Rpb24sIHN1Y2ggYXMgYWxsIHRoZSBwb3NzaWJsZSBwZXJtdXRhdGlvbnMgYmV0d2VlbiBiaW5fY29udGludW91cywgcXVhbnRpbGVfYmlucyBhbmQgdXNlX2RlbnNpdHkuIApJbiB0aGUgZXhwbGFpbiBmdW5jdGlvbiwgdGhlIG51bWJlciBvZiBwZXJtdXRhdGlvbnMgd2FzIGluY3JlYXNlZCBmcm9tIDEwIHRvIDIwMDAgYW5kIHRoZSBFdWNsaWRlYW4gYW5kIEdvd2VyIGRpc3RhbmNlIGZ1bmN0aW9ucyB3ZXJlIHRlc3RlZCB3aXRoIGtlcm5lbF93aWR0aHMgcmFuZ2luZyBmcm9tIDAuMDAxIHRvIDUgYmVjYXVzZSB3ZSB3ZXJlIG5vdCBnZXR0aW5nIGNvbnNpc3RlbnQgCgojIEYpIFNIQVAgVmFsdWVzIAoKYGBge3J9ClNIQVBfSDJPMSA8LSBoMm8ucHJlZGljdF9jb250cmlidXRpb25zKHJmaDJvLCBhcy5oMm8oc2FtcGxlRGF0YVssYygxOjcpXSkpCmhlYWQoU0hBUF9IMk8xLDEwKQpgYGAKCiMgRykgUERQIAoKCmBgYHtyfQpoMm8ucGFydGlhbFBsb3QocmZoMm8sIGRhdGEgPSBhcy5oMm8oc2x0MmRhdGFfaDJvKSwgY29scyA9ICJEaXN0YW5jZSIsIHBsb3Q9VFJVRSwgbmJpbnM9MikKCnBhcnRpYWwob3JpZ1JGLCBwcmVkLmRhdGEgPSB0cmFpbkRhdGFbLGMoMTo3KV0sIHgudmFyID0gIkRpc3RhbmNlIikKCnBhcnRpYWxQbG90KG9yaWdSRiwgcHJlZC5kYXRhID0gdHJhaW5EYXRhWyxjKDE6NyldLCB4ID0gIkRpc3RhbmNlIiApCgp0cmFpbkRhdGFbLGMoMTo5KV0KI2gyby5wYXJ0aWFsUGxvdChyZmgybywgZGF0YSA9IGFzLmgybyh0cmFpbkRhdGEpLCBjb2xzID0gIlNTIikKI2gyby5wYXJ0aWFsUGxvdChyZmgybywgZGF0YSA9IGFzLmgybyh0cmFpbkRhdGEpLCBjb2xzID0gIlBvczEwd3J0c1JOQVN0YXJ0IikKI2gyby5wYXJ0aWFsUGxvdChyZmgybywgZGF0YSA9IGFzLmgybyh0cmFpbkRhdGEpLCBjb2xzID0gIkRpc3RUZXJtIikKI2gyby5wYXJ0aWFsUGxvdChyZmgybywgZGF0YSA9IGFzLmgybyh0cmFpbkRhdGEpLCBjb2xzID0gIkRpc3RhbmNlIikKI2gyby5wYXJ0aWFsUGxvdChyZmgybywgZGF0YSA9IGFzLmgybyh0cmFpbkRhdGEpLCBjb2xzID0gIkRvd25EaXN0YW5jZSIpCiNoMm8ucGFydGlhbFBsb3QocmZoMm8sIGRhdGEgPSBhcy5oMm8odHJhaW5EYXRhKSwgY29scyA9ICJzYW1lU3RyYW5kIikKI2gyby5wYXJ0aWFsUGxvdChyZmgybywgZGF0YSA9IGFzLmgybyh0cmFpbkRhdGEpLCBjb2xzID0gInNhbWVEb3duU3RyYW5kIikKYGBgCgo=